-- Logs begin at Mon 2022-07-04 07:19:07 UTC, end at Mon 2022-07-04 08:20:53 UTC. -- Jul 04 07:48:44.103089 dv-cinder-797608-42-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 04 07:48:44.118100 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 4 07:48:44 2022] *** Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: nodename: dv-cinder-797608-42-iscsi Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: machine: x86_64 Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: clock source: unix Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: pcre jit disabled Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: detected number of CPU cores: 4 Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: current working directory: / Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: detected binary path: /usr/bin/uwsgi-core Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: your processes number limit is 47680 Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: your memory page size is 4096 bytes Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: detected max file descriptor number: 2048 Jul 04 07:48:44.120319 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: lock engine: pthread robust mutexes Jul 04 07:48:44.121728 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: thunder lock: enabled Jul 04 07:48:44.121728 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 04 07:48:44.121728 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 04 07:48:44.166026 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: Python main interpreter initialized at 0x5592ae9c59c0 Jul 04 07:48:44.166026 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: python threads support enabled Jul 04 07:48:44.166026 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: your server socket listen backlog is limited to 100 connections Jul 04 07:48:44.166026 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: your mercy for graceful operations on workers is 90 seconds Jul 04 07:48:44.166560 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: mapped 403077 bytes (393 KB) for 2 cores Jul 04 07:48:44.166560 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: *** Operational MODE: preforking *** Jul 04 07:48:44.166785 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: *** uWSGI is running in multiple interpreter mode *** Jul 04 07:48:44.166785 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: spawned uWSGI master process (pid: 80131) Jul 04 07:48:44.167055 dv-cinder-797608-42-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 04 07:48:44.168340 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: spawned uWSGI worker 1 (pid: 80143, cores: 1) Jul 04 07:48:44.168340 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: spawned uWSGI worker 2 (pid: 80144, cores: 1) Jul 04 07:48:44.168340 dv-cinder-797608-42-iscsi devstack@keystone.service[80131]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 04 07:48:49.751521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 07:48:49.751521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: from cryptography.utils import int_from_bytes Jul 04 07:48:49.752753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 07:48:49.752753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: from cryptography.utils import int_from_bytes Jul 04 07:48:49.756794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 07:48:49.756794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: from cryptography.utils import int_from_bytes Jul 04 07:48:49.757587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 07:48:49.757587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: from cryptography.utils import int_from_bytes Jul 04 07:48:49.803586 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 04 07:48:49.803844 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 04 07:48:49.803974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 04 07:48:49.804135 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 04 07:48:49.804269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] command line args: [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 04 07:48:49.804395 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] command line args: [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 04 07:48:49.804585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 04 07:48:49.804758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 04 07:48:49.804913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ================================================================================ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 04 07:48:49.805074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ================================================================================ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 04 07:48:49.805213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] admin_token = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.805361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] admin_token = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.805505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config_dir = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.805505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.805833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] config_dir = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.805986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config_source = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.805986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] debug = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.805986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.806640 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.806755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.806755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.806979 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] config_source = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.807097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.807097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.807308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] debug = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.807449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.807449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_config_append = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.807667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.807806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.807806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_dir = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.808036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.808155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.808277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.808400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.808513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.808513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.808719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.808836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.808945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.809058 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.809180 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.809303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.809432 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.809702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.809840 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_config_append = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.809962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.810080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.810196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.810376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.810580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_dir = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.810777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.810916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_file = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.811112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.811112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.811426 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.811554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.811735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.811863 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.812046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.812199 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] publish_errors = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.812199 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.812513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.812709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.812709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.813019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.813213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.813343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.813526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.813526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] standard_threads = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.813832 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.813976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.813976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.814279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.814428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.814428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_journal = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.814755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.814942 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_json = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.814942 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_stderr = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.815241 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.815424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_syslog = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.815424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.815755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.815951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.815951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.816261 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.816369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.816537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.816650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.816650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.816942 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.817062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.817062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.817287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.817414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.817414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.817414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.817841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.817984 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.817984 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.818212 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] publish_errors = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.818327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.818467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.818620 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.818620 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.818923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.819053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.819186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.819325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.819457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.819581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.819581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.819810 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.819931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.820065 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] standard_threads = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.820198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.820198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.820419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.820544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.820544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.820797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.820934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.821062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.821248 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.821459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_journal = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.821678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.821834 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_json = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.821962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.822085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_stderr = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.822215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.822215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.822454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_syslog = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.822714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.822840 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 07:48:49.822960 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.822960 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.823217 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.823340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.823340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.823575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.823699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.823823 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.823944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.823944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.824178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.824301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.824301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.824560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.824672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.824672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.824911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.825035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.825176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.825302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.825428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.825552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.825552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.825860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.825988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.825988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.826234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.826373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.826373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.826596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.826682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.826801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.826878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.826878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.827081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.827210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.827210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.827454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.827581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.827713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.827855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.827982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.828107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.828232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.828365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.828606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.828732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.828859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.828986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.828986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.829249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.829377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.829514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.829661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.829790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.829914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.830034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.830162 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.830293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.830419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.830650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.830827 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.831002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.831167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.831343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.831505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.831680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.831840 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.832012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.832012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.832327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.832486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.832656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.832829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.832829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.833213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.833355 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.833355 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.833736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.833847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.833847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.834123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.834300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.834484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.834611 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.834611 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.834878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.835046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.835046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.835289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.835427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.835427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.835632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.835745 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.835745 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.835966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.836085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.836085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.836286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.836397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.836498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.836620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.836760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.837016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.837150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.837268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.837401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.837535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.837769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.837899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.838041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.838240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.838388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.838589 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.838718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.838839 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.external = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.838970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.839094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.839308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.839442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.839570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.839689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.839812 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.password = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.839971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.token = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.840117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.840230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.840408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.840489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.840649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.840764 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.840908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.841110 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.841243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.841370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.841496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.841647 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.841804 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.841931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.842060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.842244 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.842373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.842579 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.842725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.842855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.842987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.843112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.843275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.843405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.843650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.843795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.843934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.844063 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.844215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.844311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.844380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.844597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.844736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.844869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.844998 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.845125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.845251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.845479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.845710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.845710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.845973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.846104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.846234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.846376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.846517 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.846645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.846777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.846881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.846952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.847090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.847289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.847423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.847564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.847690 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.847820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.847995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.848132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.848243 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.848421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.848559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.848690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.848814 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.848939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.849144 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.849279 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.849408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.849537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.849695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.849827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.849976 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.850106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.850267 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.850400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.850529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.850654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.850797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.850969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.851109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.851235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.851405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.851545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.851683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.851808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.851933 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.852126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.852257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.852385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.852579 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.852819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.852948 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.853169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.853305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.853435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.853714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.853834 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.853945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.854075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.854202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.854331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.854527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.854662 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.854790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.854914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.855055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.855233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.855406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.855537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.855665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.855794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.855918 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.856040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.856183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.856408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.856533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.856643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.856808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.856938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.857079 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.857203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.857331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.857455 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.857583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.857729 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.857944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.858078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.858203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.858340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.858466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.858639 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.858784 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.858910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.859076 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.859206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.859333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.859465 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.859705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.859833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.859948 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.860063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.860216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.860338 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.860452 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.860634 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.860747 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.860912 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.861050 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.861179 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.861373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.861505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.861661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.861844 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.861974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.862101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.862245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.external = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.862375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.862541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.862653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.862773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.862996 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.863143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.863273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.863398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.863539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.863666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.863800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.password = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.863926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.864143 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.864280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.token = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.864409 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.864545 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.864686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.864816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.864941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.865068 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.865268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.865403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.865537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.865748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.865872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.865991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.866108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.866226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.866419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.866538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.866667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.866787 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.866917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.867126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.867273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.867403 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.867533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.867659 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.867787 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.867951 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.868134 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.868320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.868412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.868546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.868786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.868904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.869029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.869267 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.869384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.869495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.869730 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.869874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.870021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.870150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.870279 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.870456 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.870598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.870737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.870919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.871052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.871176 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.871302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.871430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.871614 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.871761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.871888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.872077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.872211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.872338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.872533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.872686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.872815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.872943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.873071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.873240 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.873385 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.873515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.873735 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.873858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.873978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.874166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.874290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.874416 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.874539 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.874669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.874888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.875032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.875281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.875384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.875542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.875623 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.875793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.876013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.876164 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.876294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.876410 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.876537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.876721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.876860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.877008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.877205 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.877347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.877475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.877592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.877809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.877952 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.878082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.878208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.878670 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.878800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.878927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.879091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.879225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.879354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.879479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.879606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.879808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.879957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.880090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.880222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.880336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.880466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.880593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.880738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.880940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.881087 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.881216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.881343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.881474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.881644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.881769 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.881900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.882103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.882234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.882380 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.882524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.882652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.882779 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.882905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.883036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.883228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.883368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.883502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.883618 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.883867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.884086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.884207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.884332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.884560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.884697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.884829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.885012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.885148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.885385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.885822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.885953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.886072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.886191 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.886309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.886427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.886427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.886639 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.886755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.886755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.886983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.887113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.887255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.887383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.887518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.887649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.887778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.887919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.888062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.888195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.888324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.888413 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.888538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.888709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 04 07:48:49.888858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.889082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.889383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.889742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.890065 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.890370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.890686 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.891001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.891299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.891605 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.891903 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.892218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.892525 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.892839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.893151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.893457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.893803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.894113 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.894403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.894697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.895014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.895311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.895612 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.895916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.896215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.896515 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.896826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.897141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.897437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.897793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.898110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.898429 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.898734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.899033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.899505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.899819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.900128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.900421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.900734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.901059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.901384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.901816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.902138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.902431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.902749 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.903054 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.903360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.903664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.903965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.904281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.904596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.904899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.905228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.905524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.905861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.906163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.906459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.906759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.907065 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.907369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.907661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.907968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.908274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.908579 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.908894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.909193 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.909512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.909866 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.910166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.910473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.910818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.911119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.911409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] role.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.911732 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] role.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.912027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.912344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.912654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.912987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.913320 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.913722 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.913990 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.914304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.914613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.914907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.915254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.915569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.915955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.916269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.916762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.917057 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.917369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.917720 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.918043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.918377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.918681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.919007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.919322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.919635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.919935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.920236 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.920536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.920687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:49.920817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:49.920950 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.921061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:49.921321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.921679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.921979 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.922279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.922571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.922868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.923166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.923476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.923773 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.924091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.924401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.924699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.925004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.925306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.925652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.925969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.926275 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.926578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.926887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.927200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.927494 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.927806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.928167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.928501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.928813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.929131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.929431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.929788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.929920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:49.930143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.930445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.930856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.931154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.931466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.931795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.932163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.932466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.932779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.933076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.933383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.933712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.934037 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.934347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 07:48:49.934587 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 04 07:48:49.960024 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:49.960297 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:49.962901 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:49.963316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:49.963559 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:49.979912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.000213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.000489 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.115453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.139857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.168101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.168373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.176233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.176524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.236400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.236700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.236840 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.237017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.237228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.237436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.237594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.237869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.238170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.238470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.238763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.239057 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.264582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.264847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.265045 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.265311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.265529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.265758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.265892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.266034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.266244 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.266554 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.266861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.267154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.267444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.267735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.277984 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.278963 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.291685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.291940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.292080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.292080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.292321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.292490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.292490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.292736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.292872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.292872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.293127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.293260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.293552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.293887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 07:48:50.311043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.311344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 07:48:50.311570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.311711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.311836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 07:48:50.311956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 07:48:50.312092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.312241 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.312455 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.312626 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.312761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.312892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 07:48:50.313186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.313529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.313897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.314231 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.323841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.324143 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.324410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.324685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.326263 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.326637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.326984 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.327347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.333065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.335785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.336382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.336662 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.336977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.337252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.337525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.337802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.338059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.338326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.338589 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.338858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.339118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.340451 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.340843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.341229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.341782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.342168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.342532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.342902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.343285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.343664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.344043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.344441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.366194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.367730 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.367995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.372366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.373535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.373847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.374122 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.374366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.374614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.380379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.380702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.381034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.382029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.382326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.387233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.387539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.387818 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.388023 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.388303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.388569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.388825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.389091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.389830 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.390174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.394950 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.395286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.395610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.395935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.396257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.396572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.396883 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.397198 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.408014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.408376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.413061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.416591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.416862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.417074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.417283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.421772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.422349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.425681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.425997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.426210 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.429679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.429986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.430275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.430556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.431073 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.431465 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.432115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.435909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.439985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.440276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.440571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.440854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.441129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.441406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.451150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.451447 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.451715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.451979 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.459954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.460843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.461249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.461515 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.461689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.462154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.462259 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.465387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.465681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.465958 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.466231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.466594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.469889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.471621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.471902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.474995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.475274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.475548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.475826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.476103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.480269 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.480561 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.480849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.481119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.481393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.481695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.481970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.490960 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.491236 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.491515 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.491800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.492077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.492368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.492646 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.502158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.505901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.509367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.511130 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.512924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.513200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.513447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.513609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.516800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.520131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.520820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.521752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.522488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.523379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.523655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.523940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.525954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.526251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.526529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.526799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.531805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.533490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.536112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.536407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.536687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.536838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.537041 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.537317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.537592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.545020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.545310 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.545586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.545854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.546123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.546385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.546667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.546931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.547098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.547244 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.547383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.547522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.547658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.547810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.548049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.548306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.548569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.555450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 07:48:50.555908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.556189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.556461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.556728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.557017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.557287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.557561 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.557863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.558128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.558390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.558642 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.558905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.559164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 07:48:50.590898 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5592ae9c59c0 pid: 80143 (default app) Jul 04 07:48:50.594772 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80435b8-4bcc-48f0-ac44-5492277afe77 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:48:50.595058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80435b8-4bcc-48f0-ac44-5492277afe77 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:48:50.595345 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80435b8-4bcc-48f0-ac44-5492277afe77 None None] PATH_INFO: `/v3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:48:50.598162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5592ae9c59c0 pid: 80144 (default app) Jul 04 07:48:50.599904 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1/1] 10.222.0.43 () {50 vars in 791 bytes} [Mon Jul 4 07:48:50 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:48:58.764068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e091b-8b73-46df-83c2-b66a339cec15 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:48:58.764676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e091b-8b73-46df-83c2-b66a339cec15 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:48:58.764676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e091b-8b73-46df-83c2-b66a339cec15 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:48:58.769146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1/2] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:48:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:48:58.777329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5fb47-822a-483a-9d44-ac77eacc1d6e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:48:58.777951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5fb47-822a-483a-9d44-ac77eacc1d6e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:48:58.778267 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5fb47-822a-483a-9d44-ac77eacc1d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:48:58.856208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: INFO dbcounter [None req-72e5fb47-822a-483a-9d44-ac77eacc1d6e None None] Registered counter for database keystone Jul 04 07:48:58.879564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG oslo_db.sqlalchemy.engines [None req-72e5fb47-822a-483a-9d44-ac77eacc1d6e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80143) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 04 07:48:58.890108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writer thread running {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 04 07:48:59.077219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG passlib.handlers.bcrypt [None req-72e5fb47-822a-483a-9d44-ac77eacc1d6e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80143) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 04 07:48:59.077537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG passlib.handlers.bcrypt [None req-72e5fb47-822a-483a-9d44-ac77eacc1d6e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80143) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 04 07:48:59.133557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-72e5fb47-822a-483a-9d44-ac77eacc1d6e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:48:59.135282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-72e5fb47-822a-483a-9d44-ac77eacc1d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:48:59.269512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 2/3] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:48:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 04 07:48:59.487198 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e091b-8b73-46df-83c2-b66a339cec15 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:48:59.561396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: INFO dbcounter [None req-fd7e091b-8b73-46df-83c2-b66a339cec15 None None] Registered counter for database keystone Jul 04 07:48:59.582559 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG oslo_db.sqlalchemy.engines [None req-fd7e091b-8b73-46df-83c2-b66a339cec15 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80144) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 04 07:48:59.589888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writer thread running {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 04 07:48:59.723431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e091b-8b73-46df-83c2-b66a339cec15 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:48:59.728506 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ba8a9-6e7e-4096-92f4-d0ba1a71a3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:48:59.729479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ba8a9-6e7e-4096-92f4-d0ba1a71a3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:48:59.729806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ba8a9-6e7e-4096-92f4-d0ba1a71a3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:48:59.730087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ba8a9-6e7e-4096-92f4-d0ba1a71a3bd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:48:59.739782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ba8a9-6e7e-4096-92f4-d0ba1a71a3bd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:48:59.794486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: WARNING keystone.common.rbac_enforcer.enforcer [None req-279ba8a9-6e7e-4096-92f4-d0ba1a71a3bd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 04 07:48:59.795199 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ba8a9-6e7e-4096-92f4-d0ba1a71a3bd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:48:59.805670 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 2/4] 10.222.0.43 () {58 vars in 1324 bytes} [Mon Jul 4 07:48:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 07:48:59.811982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5fb47-822a-483a-9d44-ac77eacc1d6e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:48:59.829115 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5fb47-822a-483a-9d44-ac77eacc1d6e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:48:59.833883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61727f2f-126f-4642-bb4a-030056bc860d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:48:59.835095 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61727f2f-126f-4642-bb4a-030056bc860d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:48:59.835378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61727f2f-126f-4642-bb4a-030056bc860d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:48:59.835664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61727f2f-126f-4642-bb4a-030056bc860d admin admin] PATH_INFO: `/v3/projects/cf3a0790796c49f294035007b8a8c25a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:48:59.840267 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61727f2f-126f-4642-bb4a-030056bc860d admin admin] RBAC: Authorizing `identity:get_project(project_id=cf3a0790796c49f294035007b8a8c25a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:48:59.892112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: WARNING keystone.common.rbac_enforcer.enforcer [None req-61727f2f-126f-4642-bb4a-030056bc860d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 04 07:48:59.892462 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61727f2f-126f-4642-bb4a-030056bc860d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:48:59.897344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 3/5] 10.222.0.43 () {56 vars in 1207 bytes} [Mon Jul 4 07:48:59 2022] GET /identity/v3/projects/cf3a0790796c49f294035007b8a8c25a => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:01.975179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f418105-e470-4ba4-aafe-5edcd541ef4f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:01.975860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f418105-e470-4ba4-aafe-5edcd541ef4f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:01.975860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f418105-e470-4ba4-aafe-5edcd541ef4f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:01.980280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 3/6] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:49:01.986936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e98b4-b081-4690-88bf-9badacc0a3bf None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:01.987549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e98b4-b081-4690-88bf-9badacc0a3bf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:01.987549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e98b4-b081-4690-88bf-9badacc0a3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:02.008396 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c60e98b4-b081-4690-88bf-9badacc0a3bf None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:02.010141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c60e98b4-b081-4690-88bf-9badacc0a3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:02.018938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 4/7] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 04 07:49:02.046314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f418105-e470-4ba4-aafe-5edcd541ef4f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:02.060333 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f418105-e470-4ba4-aafe-5edcd541ef4f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:02.065099 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37884af-26a3-4d01-9787-023ca9611ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:02.065855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37884af-26a3-4d01-9787-023ca9611ccf admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:02.066195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37884af-26a3-4d01-9787-023ca9611ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:02.066528 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37884af-26a3-4d01-9787-023ca9611ccf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:02.075741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37884af-26a3-4d01-9787-023ca9611ccf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:02.077766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37884af-26a3-4d01-9787-023ca9611ccf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:02.088232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 4/8] 10.222.0.43 () {58 vars in 1324 bytes} [Mon Jul 4 07:49:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 07:49:02.094805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e98b4-b081-4690-88bf-9badacc0a3bf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:02.105925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e98b4-b081-4690-88bf-9badacc0a3bf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:02.111575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4559c6-8df6-489e-955c-3d6a3de344b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:02.112488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4559c6-8df6-489e-955c-3d6a3de344b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:02.112774 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4559c6-8df6-489e-955c-3d6a3de344b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:02.113054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4559c6-8df6-489e-955c-3d6a3de344b2 admin admin] PATH_INFO: `/v3/users/aec7b27e85c74490a82ad099b2e0743e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:02.114669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4559c6-8df6-489e-955c-3d6a3de344b2 admin admin] RBAC: Authorizing `identity:get_user(user_id=aec7b27e85c74490a82ad099b2e0743e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:02.116587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4559c6-8df6-489e-955c-3d6a3de344b2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:02.118516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 5/9] 10.222.0.43 () {56 vars in 1198 bytes} [Mon Jul 4 07:49:02 2022] GET /identity/v3/users/aec7b27e85c74490a82ad099b2e0743e => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:04.403249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166192f-facf-4c71-9882-3c2d5bbda259 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:04.403819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166192f-facf-4c71-9882-3c2d5bbda259 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:04.403819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166192f-facf-4c71-9882-3c2d5bbda259 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:04.405122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 5/10] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:49:04.411102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10459049-8d3c-45fd-8299-b5c095cbb3c6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:04.411677 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10459049-8d3c-45fd-8299-b5c095cbb3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:04.411677 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10459049-8d3c-45fd-8299-b5c095cbb3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:04.430218 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-10459049-8d3c-45fd-8299-b5c095cbb3c6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:04.431533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-10459049-8d3c-45fd-8299-b5c095cbb3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:04.527296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 6/11] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:04.550982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166192f-facf-4c71-9882-3c2d5bbda259 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:04.651738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a15551-700f-4b0f-8c62-e639e631c2ab None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:04.652194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a15551-700f-4b0f-8c62-e639e631c2ab None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:04.652601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a15551-700f-4b0f-8c62-e639e631c2ab None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:04.654486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 7/12] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:49:04.660920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f88d17-256f-4447-b884-6af5aa3e6f4f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:04.661364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f88d17-256f-4447-b884-6af5aa3e6f4f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:04.661810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f88d17-256f-4447-b884-6af5aa3e6f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:04.681262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-99f88d17-256f-4447-b884-6af5aa3e6f4f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:04.682604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-99f88d17-256f-4447-b884-6af5aa3e6f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:04.704022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:04.771407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 8/13] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:04.786454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166192f-facf-4c71-9882-3c2d5bbda259 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:04.814227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f88d17-256f-4447-b884-6af5aa3e6f4f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:04.826522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad5d12-0dda-481b-9571-74435c33dbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:04.827207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad5d12-0dda-481b-9571-74435c33dbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:04.827491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad5d12-0dda-481b-9571-74435c33dbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:04.827766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad5d12-0dda-481b-9571-74435c33dbc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:04.876616 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:04.914166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad5d12-0dda-481b-9571-74435c33dbc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:04.916021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad5d12-0dda-481b-9571-74435c33dbc8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:04.974708 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f88d17-256f-4447-b884-6af5aa3e6f4f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:05.036012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae34a-939c-4e7c-a7ce-faccf7d61858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:05.036370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 6/14] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:05.037415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae34a-939c-4e7c-a7ce-faccf7d61858 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:05.038014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae34a-939c-4e7c-a7ce-faccf7d61858 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:05.038440 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae34a-939c-4e7c-a7ce-faccf7d61858 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:05.043009 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad5d12-0dda-481b-9571-74435c33dbc8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:05.059766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ae34a-939c-4e7c-a7ce-faccf7d61858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ff6ae34a-939c-4e7c-a7ce-faccf7d61858 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:05.078749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:05.080140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:05.081200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 9/15] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:49:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:05.089881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae34a-939c-4e7c-a7ce-faccf7d61858 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:05.115506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:05.139869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:05.182182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad5d12-0dda-481b-9571-74435c33dbc8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:05.247207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3334948-aa55-46ca-bae5-12f5a4d7f27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:05.248424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3334948-aa55-46ca-bae5-12f5a4d7f27f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:05.248872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3334948-aa55-46ca-bae5-12f5a4d7f27f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:05.249275 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3334948-aa55-46ca-bae5-12f5a4d7f27f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:05.267098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae34a-939c-4e7c-a7ce-faccf7d61858 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:05.282473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3334948-aa55-46ca-bae5-12f5a4d7f27f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:05.285364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3334948-aa55-46ca-bae5-12f5a4d7f27f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e3334948-aa55-46ca-bae5-12f5a4d7f27f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:49:05.321308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:05.322502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:05.323390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:49:05.323390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:49:05.323390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:49:05.323390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:05.324072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 7/16] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:05.324148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f5c61-4bca-4765-8208-6538a8796646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:05.325144 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f5c61-4bca-4765-8208-6538a8796646 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:05.325687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f5c61-4bca-4765-8208-6538a8796646 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:05.326173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f5c61-4bca-4765-8208-6538a8796646 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:05.327602 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f5c61-4bca-4765-8208-6538a8796646 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:05.328153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f5c61-4bca-4765-8208-6538a8796646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:05.330864 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f5c61-4bca-4765-8208-6538a8796646 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:05.332908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3334948-aa55-46ca-bae5-12f5a4d7f27f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:05.350466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 10/17] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:05.357248 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f5c61-4bca-4765-8208-6538a8796646 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:05.457660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3334948-aa55-46ca-bae5-12f5a4d7f27f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:05.469982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:05.486931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:05.496149 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc6891-8afe-49a5-a9af-e928015aa388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:05.496838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc6891-8afe-49a5-a9af-e928015aa388 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:05.497120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc6891-8afe-49a5-a9af-e928015aa388 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:05.497390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc6891-8afe-49a5-a9af-e928015aa388 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:05.498179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc6891-8afe-49a5-a9af-e928015aa388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:05.498477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc6891-8afe-49a5-a9af-e928015aa388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:05.499924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc6891-8afe-49a5-a9af-e928015aa388 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:05.513290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f5c61-4bca-4765-8208-6538a8796646 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:05.521158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 8/18] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:05.554418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e57db-90a6-42d3-9e1f-4363013b5743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:05.555461 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e57db-90a6-42d3-9e1f-4363013b5743 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:05.555744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e57db-90a6-42d3-9e1f-4363013b5743 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:05.556037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e57db-90a6-42d3-9e1f-4363013b5743 None admin] PATH_INFO: `/v3/users/aec7b27e85c74490a82ad099b2e0743e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:05.557672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e57db-90a6-42d3-9e1f-4363013b5743 None admin] RBAC: Authorizing `identity:get_user(user_id=aec7b27e85c74490a82ad099b2e0743e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:05.559195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e57db-90a6-42d3-9e1f-4363013b5743 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:05.561232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 11/19] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:05 2022] GET /identity/v3/users/aec7b27e85c74490a82ad099b2e0743e => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:05.566846 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc6891-8afe-49a5-a9af-e928015aa388 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:05.688447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc6891-8afe-49a5-a9af-e928015aa388 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:05.727660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bca68d-a202-4a5d-901f-91d7a032f198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:05.729044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bca68d-a202-4a5d-901f-91d7a032f198 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:05.729504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bca68d-a202-4a5d-901f-91d7a032f198 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:05.729970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bca68d-a202-4a5d-901f-91d7a032f198 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:05.731836 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bca68d-a202-4a5d-901f-91d7a032f198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:05.733543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bca68d-a202-4a5d-901f-91d7a032f198 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:05.735415 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 9/20] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:05.741972 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e57db-90a6-42d3-9e1f-4363013b5743 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:05.880853 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e57db-90a6-42d3-9e1f-4363013b5743 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:05.941226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd1b6f-968a-4ea5-9a11-8f1e11bdda95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:05.942336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd1b6f-968a-4ea5-9a11-8f1e11bdda95 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:05.942767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd1b6f-968a-4ea5-9a11-8f1e11bdda95 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:05.943194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd1b6f-968a-4ea5-9a11-8f1e11bdda95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:05.944631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd1b6f-968a-4ea5-9a11-8f1e11bdda95 None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.domain.id=default, user.id=aec7b27e85c74490a82ad099b2e0743e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:05.945093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd1b6f-968a-4ea5-9a11-8f1e11bdda95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:05.947139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd1b6f-968a-4ea5-9a11-8f1e11bdda95 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:05.956929 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 12/21] 10.222.0.43 () {56 vars in 1322 bytes} [Mon Jul 4 07:49:05 2022] GET /identity/v3/role_assignments?user.id=aec7b27e85c74490a82ad099b2e0743e&scope.domain.id=default&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 233 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:08.622992 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca34a0-b22b-4470-8685-1d3e50689de7 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:08.624303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca34a0-b22b-4470-8685-1d3e50689de7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:08.624917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca34a0-b22b-4470-8685-1d3e50689de7 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:08.627323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 10/22] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:08.636945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efb7b2-ad0f-4111-9938-1927d454b4cf None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:08.638222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efb7b2-ad0f-4111-9938-1927d454b4cf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:08.638809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efb7b2-ad0f-4111-9938-1927d454b4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:08.662776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:08.675472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-50efb7b2-ad0f-4111-9938-1927d454b4cf None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:08.677438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-50efb7b2-ad0f-4111-9938-1927d454b4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:08.838841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 13/23] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:08.917686 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca34a0-b22b-4470-8685-1d3e50689de7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:09.011305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:09.077526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca34a0-b22b-4470-8685-1d3e50689de7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:09.125682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff966c5-db03-4584-86a0-6189b8d1f720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:09.125682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff966c5-db03-4584-86a0-6189b8d1f720 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:09.125682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff966c5-db03-4584-86a0-6189b8d1f720 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:09.125682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff966c5-db03-4584-86a0-6189b8d1f720 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:09.146783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff966c5-db03-4584-86a0-6189b8d1f720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-aff966c5-db03-4584-86a0-6189b8d1f720 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:09.156045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:09.157250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:09.158423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 11/24] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:49:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:49:09.165489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efb7b2-ad0f-4111-9938-1927d454b4cf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:09.304697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efb7b2-ad0f-4111-9938-1927d454b4cf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:09.344063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4f1fc-b168-4d79-b5f2-933cdfc60de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:09.344943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4f1fc-b168-4d79-b5f2-933cdfc60de9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:09.345235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4f1fc-b168-4d79-b5f2-933cdfc60de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:09.345575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4f1fc-b168-4d79-b5f2-933cdfc60de9 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:09.346379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4f1fc-b168-4d79-b5f2-933cdfc60de9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:09.346733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4f1fc-b168-4d79-b5f2-933cdfc60de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:09.348198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4f1fc-b168-4d79-b5f2-933cdfc60de9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:09.351304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:09.354150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c96fea-2de0-4a1e-ba8f-22d744d35111 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:09.354457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c96fea-2de0-4a1e-ba8f-22d744d35111 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:09.354709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c96fea-2de0-4a1e-ba8f-22d744d35111 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:09.356117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 12/25] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:09.361149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 14/26] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:09.367276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c96fea-2de0-4a1e-ba8f-22d744d35111 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:09.369806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b0601-227c-4b40-bfb2-02ba2f4dd53f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:09.370104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b0601-227c-4b40-bfb2-02ba2f4dd53f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:09.370388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b0601-227c-4b40-bfb2-02ba2f4dd53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:09.411121 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f18b0601-227c-4b40-bfb2-02ba2f4dd53f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:09.412644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f18b0601-227c-4b40-bfb2-02ba2f4dd53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:09.544603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 15/27] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:09.546141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb25d9-ac4f-4a49-9bf3-5ea3b23f6028 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:09.546431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb25d9-ac4f-4a49-9bf3-5ea3b23f6028 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:09.546704 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb25d9-ac4f-4a49-9bf3-5ea3b23f6028 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:09.548277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 16/28] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:09.553765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0c118-07b1-4284-bf90-60f6dc6bb1bf None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:09.554052 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0c118-07b1-4284-bf90-60f6dc6bb1bf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:09.554335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0c118-07b1-4284-bf90-60f6dc6bb1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:09.583245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-15e0c118-07b1-4284-bf90-60f6dc6bb1bf None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:09.584377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-15e0c118-07b1-4284-bf90-60f6dc6bb1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:09.600180 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c96fea-2de0-4a1e-ba8f-22d744d35111 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:09.630005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:09.675892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fe34f-3d44-4387-8070-7a2f37d9caa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:09.676789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fe34f-3d44-4387-8070-7a2f37d9caa6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:09.677069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fe34f-3d44-4387-8070-7a2f37d9caa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:09.677360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fe34f-3d44-4387-8070-7a2f37d9caa6 None admin] PATH_INFO: `/v3/users/aec7b27e85c74490a82ad099b2e0743e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:09.682212 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fe34f-3d44-4387-8070-7a2f37d9caa6 None admin] RBAC: Authorizing `identity:get_user(user_id=aec7b27e85c74490a82ad099b2e0743e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:09.683724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fe34f-3d44-4387-8070-7a2f37d9caa6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:09.685753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 13/29] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:09 2022] GET /identity/v3/users/aec7b27e85c74490a82ad099b2e0743e => generated 247 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:09.686271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fe34f-3d44-4387-8070-7a2f37d9caa6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:09.727004 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 17/30] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:09.727982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0c118-07b1-4284-bf90-60f6dc6bb1bf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:09.843504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fe34f-3d44-4387-8070-7a2f37d9caa6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:09.848376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:09.887270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6616f6-676a-46e9-a221-700a47de606f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:09.888427 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6616f6-676a-46e9-a221-700a47de606f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:09.888892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6616f6-676a-46e9-a221-700a47de606f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:09.889361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6616f6-676a-46e9-a221-700a47de606f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:09.902888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6616f6-676a-46e9-a221-700a47de606f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:09.904421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6616f6-676a-46e9-a221-700a47de606f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:09.909441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0c118-07b1-4284-bf90-60f6dc6bb1bf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bd6616f6-676a-46e9-a221-700a47de606f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:49:09.924542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:09.925476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:09.926505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:49:09.926505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:49:09.926505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:49:09.926505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:09.926505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 14/31] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:09.927215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6616f6-676a-46e9-a221-700a47de606f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:09.954491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b7179-1a2e-40b1-afa0-21828d47e89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:09.955311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b7179-1a2e-40b1-afa0-21828d47e89d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:09.955590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b7179-1a2e-40b1-afa0-21828d47e89d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:09.955862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b7179-1a2e-40b1-afa0-21828d47e89d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:09.957117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b7179-1a2e-40b1-afa0-21828d47e89d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:09.958752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b7179-1a2e-40b1-afa0-21828d47e89d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:09.960816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 18/32] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:09.962215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fdc2c-ed0b-40b7-bc32-ba1736e497e9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:09.962467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fdc2c-ed0b-40b7-bc32-ba1736e497e9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:09.962745 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fdc2c-ed0b-40b7-bc32-ba1736e497e9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:09.964344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 19/33] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:09.964920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0fdc2c-ed0b-40b7-bc32-ba1736e497e9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:09.972862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:10.057926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6616f6-676a-46e9-a221-700a47de606f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:10.083459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0fdc2c-ed0b-40b7-bc32-ba1736e497e9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:10.113601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:10.124664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d8b15-8f8e-44b3-8c81-7a0c4fb9c5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:10.125707 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d8b15-8f8e-44b3-8c81-7a0c4fb9c5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:10.126150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d8b15-8f8e-44b3-8c81-7a0c4fb9c5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:10.126699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d8b15-8f8e-44b3-8c81-7a0c4fb9c5e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:10.127604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d8b15-8f8e-44b3-8c81-7a0c4fb9c5e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:10.128039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d8b15-8f8e-44b3-8c81-7a0c4fb9c5e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:10.130390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d8b15-8f8e-44b3-8c81-7a0c4fb9c5e8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:10.131766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59217540-881a-49a6-88de-e4fe569cf2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:10.133181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59217540-881a-49a6-88de-e4fe569cf2b7 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:10.135076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59217540-881a-49a6-88de-e4fe569cf2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:10.135855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59217540-881a-49a6-88de-e4fe569cf2b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:10.138345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59217540-881a-49a6-88de-e4fe569cf2b7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:10.141485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59217540-881a-49a6-88de-e4fe569cf2b7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:10.154293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 20/34] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:10.155065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d8b15-8f8e-44b3-8c81-7a0c4fb9c5e8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:10.274175 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d8b15-8f8e-44b3-8c81-7a0c4fb9c5e8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:10.287153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 15/35] 10.222.0.43 () {60 vars in 1131 bytes} [Mon Jul 4 07:49:09 2022] POST /identity/v3/roles => generated 219 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:49:10.289087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a49d3-cd15-4a2c-a9fd-dd89eb68c0aa None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:10.289917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a49d3-cd15-4a2c-a9fd-dd89eb68c0aa None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:10.291128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a49d3-cd15-4a2c-a9fd-dd89eb68c0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:10.311713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG passlib.handlers.bcrypt [None req-e12a49d3-cd15-4a2c-a9fd-dd89eb68c0aa None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80144) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 04 07:49:10.313225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG passlib.handlers.bcrypt [None req-e12a49d3-cd15-4a2c-a9fd-dd89eb68c0aa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80144) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 04 07:49:10.318509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b17a1e-a96f-4b77-b849-aac7e9c8a3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:10.319480 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b17a1e-a96f-4b77-b849-aac7e9c8a3f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:10.319767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b17a1e-a96f-4b77-b849-aac7e9c8a3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:10.320048 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b17a1e-a96f-4b77-b849-aac7e9c8a3f5 None admin] PATH_INFO: `/v3/domains/default/users/aec7b27e85c74490a82ad099b2e0743e/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:10.322718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b17a1e-a96f-4b77-b849-aac7e9c8a3f5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=aec7b27e85c74490a82ad099b2e0743e, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:10.324281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b17a1e-a96f-4b77-b849-aac7e9c8a3f5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:10.364872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e12a49d3-cd15-4a2c-a9fd-dd89eb68c0aa None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:10.367031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e12a49d3-cd15-4a2c-a9fd-dd89eb68c0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:10.459208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 21/36] 10.222.0.43 () {58 vars in 1361 bytes} [Mon Jul 4 07:49:10 2022] PUT /identity/v3/domains/default/users/aec7b27e85c74490a82ad099b2e0743e/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:49:10.463558 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b17a1e-a96f-4b77-b849-aac7e9c8a3f5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:10.473416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 16/37] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:10.475610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7561f9-eb0a-49ff-8a23-c3053d138398 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:10.476098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7561f9-eb0a-49ff-8a23-c3053d138398 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:10.476553 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7561f9-eb0a-49ff-8a23-c3053d138398 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:10.478352 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 17/38] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:10.479774 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fe33d-553f-45b7-8815-28b4324a4989 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:10.480143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fe33d-553f-45b7-8815-28b4324a4989 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:10.480438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fe33d-553f-45b7-8815-28b4324a4989 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:10.481801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 18/39] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:10.485312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323a606-5b11-434c-8eb7-1509e851863d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:10.485653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323a606-5b11-434c-8eb7-1509e851863d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:10.486056 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323a606-5b11-434c-8eb7-1509e851863d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:10.514351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e323a606-5b11-434c-8eb7-1509e851863d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:10.515640 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e323a606-5b11-434c-8eb7-1509e851863d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:10.529963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:10.544494 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:10.619480 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b17a1e-a96f-4b77-b849-aac7e9c8a3f5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:10.641969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 19/40] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:10.645172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2477ad2-2943-4ed2-84bf-be9b4099568e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:10.645743 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2477ad2-2943-4ed2-84bf-be9b4099568e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:10.646252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2477ad2-2943-4ed2-84bf-be9b4099568e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:10.674926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d2477ad2-2943-4ed2-84bf-be9b4099568e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:10.676250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d2477ad2-2943-4ed2-84bf-be9b4099568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:10.689822 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1a242-da2b-43c1-845b-ab1228643fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:10.691147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1a242-da2b-43c1-845b-ab1228643fbb None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:10.691392 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1a242-da2b-43c1-845b-ab1228643fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:10.692049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1a242-da2b-43c1-845b-ab1228643fbb None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:10.693158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1a242-da2b-43c1-845b-ab1228643fbb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:10.695268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1a242-da2b-43c1-845b-ab1228643fbb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:10.752551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 22/41] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:49:10 2022] POST /identity/v3/projects => generated 301 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:49:10.754409 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1a242-da2b-43c1-845b-ab1228643fbb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:10.775152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 20/42] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:10.776071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2477ad2-2943-4ed2-84bf-be9b4099568e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:10.941227 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:10.961305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2477ad2-2943-4ed2-84bf-be9b4099568e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:11.029273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8b9fb-af86-4079-80dd-a29e45490491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:11.031128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8b9fb-af86-4079-80dd-a29e45490491 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:11.031973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8b9fb-af86-4079-80dd-a29e45490491 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:11.032770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8b9fb-af86-4079-80dd-a29e45490491 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:11.034679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8b9fb-af86-4079-80dd-a29e45490491 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:11.037728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8b9fb-af86-4079-80dd-a29e45490491 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:11.062048 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 21/43] 10.222.0.43 () {60 vars in 1131 bytes} [Mon Jul 4 07:49:10 2022] POST /identity/v3/roles => generated 223 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:49:11.069973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8b9fb-af86-4079-80dd-a29e45490491 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:11.128930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:11.135761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1a242-da2b-43c1-845b-ab1228643fbb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:11.208685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e1c84-a058-46bb-9d89-b9ffe9e9edea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:11.209275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e1c84-a058-46bb-9d89-b9ffe9e9edea None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:11.209567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e1c84-a058-46bb-9d89-b9ffe9e9edea None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:11.209882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e1c84-a058-46bb-9d89-b9ffe9e9edea None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:11.210704 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e1c84-a058-46bb-9d89-b9ffe9e9edea None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:11.212277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e1c84-a058-46bb-9d89-b9ffe9e9edea None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:11.227480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8b9fb-af86-4079-80dd-a29e45490491 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:11.238506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 23/44] 10.222.0.43 () {60 vars in 1131 bytes} [Mon Jul 4 07:49:10 2022] POST /identity/v3/roles => generated 225 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:49:11.302169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3dcf17-faba-4b8d-bac3-525e44c3485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:11.303407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dcf17-faba-4b8d-bac3-525e44c3485c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:11.303707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dcf17-faba-4b8d-bac3-525e44c3485c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:11.303982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dcf17-faba-4b8d-bac3-525e44c3485c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:11.309992 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3dcf17-faba-4b8d-bac3-525e44c3485c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:11.311585 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3dcf17-faba-4b8d-bac3-525e44c3485c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:11.313479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 22/45] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:11.321758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e1c84-a058-46bb-9d89-b9ffe9e9edea None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:11.473275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e1c84-a058-46bb-9d89-b9ffe9e9edea None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:11.515265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d815627-4297-497e-af5c-4b4f9eda6e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:11.515955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d815627-4297-497e-af5c-4b4f9eda6e9c None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:11.516241 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d815627-4297-497e-af5c-4b4f9eda6e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:11.516526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d815627-4297-497e-af5c-4b4f9eda6e9c None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:11.517267 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d815627-4297-497e-af5c-4b4f9eda6e9c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:11.518824 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d815627-4297-497e-af5c-4b4f9eda6e9c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:11.550493 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 24/46] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:49:11 2022] POST /identity/v3/projects => generated 312 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 07:49:12.670680 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4f288-e914-4c82-9544-1ac3a02cfa03 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:12.671478 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4f288-e914-4c82-9544-1ac3a02cfa03 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:12.671688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4f288-e914-4c82-9544-1ac3a02cfa03 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:12.674539 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 23/47] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:12.682666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082ad6b-5033-4195-8b35-509020026f0d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:12.683253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082ad6b-5033-4195-8b35-509020026f0d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:12.683527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082ad6b-5033-4195-8b35-509020026f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:12.712415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6082ad6b-5033-4195-8b35-509020026f0d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:12.715031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6082ad6b-5033-4195-8b35-509020026f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:12.724163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:12.810422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 25/48] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:12.834784 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4f288-e914-4c82-9544-1ac3a02cfa03 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:12.853447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:12.964440 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4f288-e914-4c82-9544-1ac3a02cfa03 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:13.007118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c08af-3903-4a73-94b6-980718aa92f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:13.008096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c08af-3903-4a73-94b6-980718aa92f7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:13.008516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c08af-3903-4a73-94b6-980718aa92f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:13.008885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c08af-3903-4a73-94b6-980718aa92f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:13.024667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c08af-3903-4a73-94b6-980718aa92f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-538c08af-3903-4a73-94b6-980718aa92f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:13.032839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:13.034614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:13.036044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 24/49] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:49:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:13.042917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082ad6b-5033-4195-8b35-509020026f0d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:13.171265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082ad6b-5033-4195-8b35-509020026f0d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:13.213897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3cb3e-39de-49a8-a51c-05d1ae510666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:13.214773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3cb3e-39de-49a8-a51c-05d1ae510666 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:13.215064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3cb3e-39de-49a8-a51c-05d1ae510666 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:13.215415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3cb3e-39de-49a8-a51c-05d1ae510666 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:13.216263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3cb3e-39de-49a8-a51c-05d1ae510666 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:13.216640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3cb3e-39de-49a8-a51c-05d1ae510666 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:13.218637 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3cb3e-39de-49a8-a51c-05d1ae510666 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:13.229349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 26/50] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:13.236145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c08af-3903-4a73-94b6-980718aa92f7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:13.373981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:13.433421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c08af-3903-4a73-94b6-980718aa92f7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:13.497820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be906f-4188-4cde-acc0-9233a752de44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:13.499276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be906f-4188-4cde-acc0-9233a752de44 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:13.499815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be906f-4188-4cde-acc0-9233a752de44 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:13.500303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be906f-4188-4cde-acc0-9233a752de44 None admin] PATH_INFO: `/v3/users/aec7b27e85c74490a82ad099b2e0743e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:13.503020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be906f-4188-4cde-acc0-9233a752de44 None admin] RBAC: Authorizing `identity:get_user(user_id=aec7b27e85c74490a82ad099b2e0743e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:13.506087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be906f-4188-4cde-acc0-9233a752de44 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:13.509417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 25/51] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:13 2022] GET /identity/v3/users/aec7b27e85c74490a82ad099b2e0743e => generated 247 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:13.517972 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3cb3e-39de-49a8-a51c-05d1ae510666 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:13.521457 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:13.638251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3cb3e-39de-49a8-a51c-05d1ae510666 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:13.676444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4645f-c215-4451-9157-dd44283490fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:13.677295 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4645f-c215-4451-9157-dd44283490fc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:13.677545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4645f-c215-4451-9157-dd44283490fc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:13.677849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4645f-c215-4451-9157-dd44283490fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:13.679109 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4645f-c215-4451-9157-dd44283490fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:13.680721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4645f-c215-4451-9157-dd44283490fc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:13.682470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 27/52] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:13.688565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be906f-4188-4cde-acc0-9233a752de44 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:13.711474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b944ee0-1e87-40c5-be60-ff0f973098f6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:13.712094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b944ee0-1e87-40c5-be60-ff0f973098f6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:13.712509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b944ee0-1e87-40c5-be60-ff0f973098f6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:13.715206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 28/53] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:49:13.723906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5e2f2-9163-40fc-b977-c93d44163dbc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:13.724408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5e2f2-9163-40fc-b977-c93d44163dbc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:13.724909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5e2f2-9163-40fc-b977-c93d44163dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:13.753321 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3aa5e2f2-9163-40fc-b977-c93d44163dbc None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:13.755770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3aa5e2f2-9163-40fc-b977-c93d44163dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:13.805445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be906f-4188-4cde-acc0-9233a752de44 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:13.848224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:13.855094 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168805d6-fa95-42aa-bde6-368f1d62f9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:13.856050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 29/54] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:13.857983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168805d6-fa95-42aa-bde6-368f1d62f9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:13.858405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168805d6-fa95-42aa-bde6-368f1d62f9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:13.858771 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168805d6-fa95-42aa-bde6-368f1d62f9a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:13.859751 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168805d6-fa95-42aa-bde6-368f1d62f9a6 None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.domain.id=default, user.id=aec7b27e85c74490a82ad099b2e0743e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:13.860170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168805d6-fa95-42aa-bde6-368f1d62f9a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:13.861828 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168805d6-fa95-42aa-bde6-368f1d62f9a6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:13.872602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 26/55] 10.222.0.43 () {56 vars in 1322 bytes} [Mon Jul 4 07:49:13 2022] GET /identity/v3/role_assignments?user.id=aec7b27e85c74490a82ad099b2e0743e&scope.domain.id=default&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 530 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:13.882724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5e2f2-9163-40fc-b977-c93d44163dbc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:14.048905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5e2f2-9163-40fc-b977-c93d44163dbc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:14.091069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beb711-c539-4151-81b2-a6c9eba182d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:14.092067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beb711-c539-4151-81b2-a6c9eba182d8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:14.092450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beb711-c539-4151-81b2-a6c9eba182d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:14.092810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beb711-c539-4151-81b2-a6c9eba182d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:14.094634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7beb711-c539-4151-81b2-a6c9eba182d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:14.096728 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7beb711-c539-4151-81b2-a6c9eba182d8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:14.098646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 30/56] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:14.106741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168805d6-fa95-42aa-bde6-368f1d62f9a6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:14.112668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:14.339518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168805d6-fa95-42aa-bde6-368f1d62f9a6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:14.377322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2038b1-b3bf-4215-bc5b-abad1a842ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:14.378062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2038b1-b3bf-4215-bc5b-abad1a842ee4 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:14.378341 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2038b1-b3bf-4215-bc5b-abad1a842ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:14.378965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2038b1-b3bf-4215-bc5b-abad1a842ee4 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:14.379710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2038b1-b3bf-4215-bc5b-abad1a842ee4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:14.381221 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2038b1-b3bf-4215-bc5b-abad1a842ee4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:14.412791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 27/57] 10.222.0.43 () {60 vars in 1140 bytes} [Mon Jul 4 07:49:14 2022] POST /identity/v3/projects => generated 298 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 07:49:16.642562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a852d-5ca7-48d8-aad9-98388c2815bd None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:16.643128 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a852d-5ca7-48d8-aad9-98388c2815bd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:16.643128 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a852d-5ca7-48d8-aad9-98388c2815bd None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:16.644529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 31/58] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:16.650553 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd72a17-b1a8-4cf0-a6aa-949207a5a280 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:16.651009 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd72a17-b1a8-4cf0-a6aa-949207a5a280 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:16.651125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd72a17-b1a8-4cf0-a6aa-949207a5a280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:16.670700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ccd72a17-b1a8-4cf0-a6aa-949207a5a280 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:16.671995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ccd72a17-b1a8-4cf0-a6aa-949207a5a280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:16.729123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:16.747303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 28/59] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:16.773515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a852d-5ca7-48d8-aad9-98388c2815bd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:16.847248 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:16.887230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a852d-5ca7-48d8-aad9-98388c2815bd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:16.926564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba3f5e-28dc-4d9a-a2fa-e1498034022b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:16.927290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba3f5e-28dc-4d9a-a2fa-e1498034022b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:16.927579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba3f5e-28dc-4d9a-a2fa-e1498034022b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:16.927854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba3f5e-28dc-4d9a-a2fa-e1498034022b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:16.929075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba3f5e-28dc-4d9a-a2fa-e1498034022b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:16.931248 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba3f5e-28dc-4d9a-a2fa-e1498034022b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:16.932953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 32/60] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:16.938790 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd72a17-b1a8-4cf0-a6aa-949207a5a280 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:17.046772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd72a17-b1a8-4cf0-a6aa-949207a5a280 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:17.090270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f032dcd2-098d-49a2-9ac1-620ae72fb08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:17.090988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032dcd2-098d-49a2-9ac1-620ae72fb08f None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:17.091279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032dcd2-098d-49a2-9ac1-620ae72fb08f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:17.091564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032dcd2-098d-49a2-9ac1-620ae72fb08f None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:17.092343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f032dcd2-098d-49a2-9ac1-620ae72fb08f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:17.093967 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f032dcd2-098d-49a2-9ac1-620ae72fb08f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:17.123393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 29/61] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:49:16 2022] POST /identity/v3/users => generated 275 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:49:20.510930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf8dcc-0c27-4452-9e3a-f32dbfb8a5a8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:20.511996 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf8dcc-0c27-4452-9e3a-f32dbfb8a5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:20.512528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf8dcc-0c27-4452-9e3a-f32dbfb8a5a8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:20.514207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 33/62] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:20.520083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c25bc-dde6-4bd8-841c-455463dac8bd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:20.521013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c25bc-dde6-4bd8-841c-455463dac8bd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:20.521505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c25bc-dde6-4bd8-841c-455463dac8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:20.543337 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9a7c25bc-dde6-4bd8-841c-455463dac8bd None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:20.545123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9a7c25bc-dde6-4bd8-841c-455463dac8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:20.608719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:20.666997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 30/63] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:20.708986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf8dcc-0c27-4452-9e3a-f32dbfb8a5a8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:20.882265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf8dcc-0c27-4452-9e3a-f32dbfb8a5a8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:20.909198 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79841d8-b2ab-4c8d-a7d8-94add02f3a9f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:20.909880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79841d8-b2ab-4c8d-a7d8-94add02f3a9f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:20.910369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79841d8-b2ab-4c8d-a7d8-94add02f3a9f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:20.914858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 31/64] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:20.931368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792a7d9-5b53-4275-9fce-2925dad3f6c8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:20.931998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792a7d9-5b53-4275-9fce-2925dad3f6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:20.932483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792a7d9-5b53-4275-9fce-2925dad3f6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:20.941308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9257b3e-9103-4509-b89c-d14183a92a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:20.942565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9257b3e-9103-4509-b89c-d14183a92a01 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:20.943057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9257b3e-9103-4509-b89c-d14183a92a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:20.943533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9257b3e-9103-4509-b89c-d14183a92a01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:20.958678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:20.961551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9792a7d9-5b53-4275-9fce-2925dad3f6c8 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:20.962872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9792a7d9-5b53-4275-9fce-2925dad3f6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:20.971380 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9257b3e-9103-4509-b89c-d14183a92a01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c9257b3e-9103-4509-b89c-d14183a92a01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:20.985160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:20.986817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:20.988956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:20.988956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:20.988956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:20.988956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:20.988956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:20.988956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:20.988956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:20.988956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:20.988956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:20.988956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:20.988956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:20.988956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:20.993669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 34/65] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:20.993669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9257b3e-9103-4509-b89c-d14183a92a01 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:21.124096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 32/66] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:21.152922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73fd35-16ee-48dd-bae4-cbb9d57f4860 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:21.153143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73fd35-16ee-48dd-bae4-cbb9d57f4860 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:21.153444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73fd35-16ee-48dd-bae4-cbb9d57f4860 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:21.154811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 33/67] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:21.169343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73fd35-16ee-48dd-bae4-cbb9d57f4860 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:21.222842 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9257b3e-9103-4509-b89c-d14183a92a01 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:21.266112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcb47a-2eba-42b4-a15c-23b6d7a7cc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:21.267279 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcb47a-2eba-42b4-a15c-23b6d7a7cc93 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:21.267797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcb47a-2eba-42b4-a15c-23b6d7a7cc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:21.268302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcb47a-2eba-42b4-a15c-23b6d7a7cc93 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:21.269783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcb47a-2eba-42b4-a15c-23b6d7a7cc93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:21.270334 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcb47a-2eba-42b4-a15c-23b6d7a7cc93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:21.273112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcb47a-2eba-42b4-a15c-23b6d7a7cc93 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:21.290709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 35/68] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:21.293178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f296139-b4d3-4303-9394-3d2f5f39f276 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:21.293735 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f296139-b4d3-4303-9394-3d2f5f39f276 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:21.294259 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f296139-b4d3-4303-9394-3d2f5f39f276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:21.312249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73fd35-16ee-48dd-bae4-cbb9d57f4860 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:21.315982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:21.317039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3f296139-b4d3-4303-9394-3d2f5f39f276 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:21.318322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3f296139-b4d3-4303-9394-3d2f5f39f276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:21.372736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08b6bb-c49c-468b-acd4-22a5b4a77467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:21.373701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08b6bb-c49c-468b-acd4-22a5b4a77467 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:21.373987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08b6bb-c49c-468b-acd4-22a5b4a77467 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:21.374256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08b6bb-c49c-468b-acd4-22a5b4a77467 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:21.384170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:21.391509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08b6bb-c49c-468b-acd4-22a5b4a77467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:21.400501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 36/69] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-3c08b6bb-c49c-468b-acd4-22a5b4a77467 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:21.402380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:21.404300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:21.406070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:21.406070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:21.406070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:21.406070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:21.406070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:21.406070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:21.406070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:21.406070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:21.406070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:21.406070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:21.406070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:21.406070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:21.407410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9c5e7-e017-445c-aa49-90cae7489f2b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:21.407410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9c5e7-e017-445c-aa49-90cae7489f2b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:21.408122 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9c5e7-e017-445c-aa49-90cae7489f2b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:21.422675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 37/70] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:21.422675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9c5e7-e017-445c-aa49-90cae7489f2b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:21.424134 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 34/71] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:49:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:21.426736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127944d-818e-4cec-912e-04fdf41a9870 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:21.427594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127944d-818e-4cec-912e-04fdf41a9870 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:21.428272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127944d-818e-4cec-912e-04fdf41a9870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:21.458752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5127944d-818e-4cec-912e-04fdf41a9870 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:21.461137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5127944d-818e-4cec-912e-04fdf41a9870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:21.548322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 35/72] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:21.554382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127944d-818e-4cec-912e-04fdf41a9870 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:21.586630 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9c5e7-e017-445c-aa49-90cae7489f2b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:21.651506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33235c3c-4e95-49af-a636-09b44ae710c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:21.653047 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33235c3c-4e95-49af-a636-09b44ae710c0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:21.653538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33235c3c-4e95-49af-a636-09b44ae710c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:21.654069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33235c3c-4e95-49af-a636-09b44ae710c0 None admin] PATH_INFO: `/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:21.654668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:21.673434 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127944d-818e-4cec-912e-04fdf41a9870 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:21.676226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33235c3c-4e95-49af-a636-09b44ae710c0 None admin] RBAC: Authorizing `identity:get_user(user_id=6c3cd73e4b2b4c3e83764570cbced9fa)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:21.678542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33235c3c-4e95-49af-a636-09b44ae710c0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:21.681672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 38/73] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:21 2022] GET /identity/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:21.683308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33235c3c-4e95-49af-a636-09b44ae710c0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:21.722066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f974073-8e4a-4748-bfc2-8c6a08d8aa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:21.723014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f974073-8e4a-4748-bfc2-8c6a08d8aa94 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:21.723378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f974073-8e4a-4748-bfc2-8c6a08d8aa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:21.723747 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f974073-8e4a-4748-bfc2-8c6a08d8aa94 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:21.735865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f974073-8e4a-4748-bfc2-8c6a08d8aa94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:21.738464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5f974073-8e4a-4748-bfc2-8c6a08d8aa94 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:21.744493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:21.746316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 36/74] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:49:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:21.747779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f974073-8e4a-4748-bfc2-8c6a08d8aa94 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:21.804549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33235c3c-4e95-49af-a636-09b44ae710c0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:21.842584 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91384bc-3aa9-4ec4-8f49-e62a4d8f7ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:21.843328 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91384bc-3aa9-4ec4-8f49-e62a4d8f7ded None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:21.843665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91384bc-3aa9-4ec4-8f49-e62a4d8f7ded None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:21.843963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91384bc-3aa9-4ec4-8f49-e62a4d8f7ded None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:21.844779 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91384bc-3aa9-4ec4-8f49-e62a4d8f7ded None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:21.845096 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91384bc-3aa9-4ec4-8f49-e62a4d8f7ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:21.846595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91384bc-3aa9-4ec4-8f49-e62a4d8f7ded None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:21.856481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 39/75] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:21.857391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91384bc-3aa9-4ec4-8f49-e62a4d8f7ded None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:21.861550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f974073-8e4a-4748-bfc2-8c6a08d8aa94 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:21.902270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a940be8-aa5f-42a6-857b-6a6f2d8536b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:21.903118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a940be8-aa5f-42a6-857b-6a6f2d8536b9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:21.903408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a940be8-aa5f-42a6-857b-6a6f2d8536b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:21.903689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a940be8-aa5f-42a6-857b-6a6f2d8536b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:21.916879 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a940be8-aa5f-42a6-857b-6a6f2d8536b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7a940be8-aa5f-42a6-857b-6a6f2d8536b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:21.924786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:21.926438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 37/76] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:21.927972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc9d29-1176-403c-80ca-6844ded1476f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:21.929270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc9d29-1176-403c-80ca-6844ded1476f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:21.929270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc9d29-1176-403c-80ca-6844ded1476f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:21.930185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 38/77] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:21.930524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc9d29-1176-403c-80ca-6844ded1476f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:21.948225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:21.990253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91384bc-3aa9-4ec4-8f49-e62a4d8f7ded None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:22.003094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:22.034841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d898360-5c38-4618-b7f0-0c2494b010ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:22.035723 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d898360-5c38-4618-b7f0-0c2494b010ce None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:22.036163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d898360-5c38-4618-b7f0-0c2494b010ce None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:22.036566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d898360-5c38-4618-b7f0-0c2494b010ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:22.045297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc9d29-1176-403c-80ca-6844ded1476f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:22.094379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd88b9-bba0-4091-b189-c09762d08a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:22.095077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd88b9-bba0-4091-b189-c09762d08a7f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:22.095364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd88b9-bba0-4091-b189-c09762d08a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:22.095645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd88b9-bba0-4091-b189-c09762d08a7f None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:22.096427 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd88b9-bba0-4091-b189-c09762d08a7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:22.096746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd88b9-bba0-4091-b189-c09762d08a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:22.098217 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd88b9-bba0-4091-b189-c09762d08a7f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:22.112112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 39/78] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:49:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:22.112880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd88b9-bba0-4091-b189-c09762d08a7f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:22.144947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d898360-5c38-4618-b7f0-0c2494b010ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:22.146517 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d898360-5c38-4618-b7f0-0c2494b010ce None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:22.238680 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:22.263862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 40/79] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:22.265113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d898360-5c38-4618-b7f0-0c2494b010ce None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:22.279840 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:22.283840 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd88b9-bba0-4091-b189-c09762d08a7f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:22.329254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee81ab8-493b-454b-9cc9-f243ae19d65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:22.330198 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee81ab8-493b-454b-9cc9-f243ae19d65e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:22.330484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee81ab8-493b-454b-9cc9-f243ae19d65e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:22.330764 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee81ab8-493b-454b-9cc9-f243ae19d65e None admin] PATH_INFO: `/v3/users/aec7b27e85c74490a82ad099b2e0743e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:22.332080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee81ab8-493b-454b-9cc9-f243ae19d65e None admin] RBAC: Authorizing `identity:get_user(user_id=aec7b27e85c74490a82ad099b2e0743e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:22.333603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee81ab8-493b-454b-9cc9-f243ae19d65e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:22.335431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 40/80] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:22 2022] GET /identity/v3/users/aec7b27e85c74490a82ad099b2e0743e => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:22.337019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b56d7f-8393-4085-a85f-d21c178b911a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:22.337315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b56d7f-8393-4085-a85f-d21c178b911a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:22.337600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b56d7f-8393-4085-a85f-d21c178b911a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:22.355853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-57b56d7f-8393-4085-a85f-d21c178b911a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:22.357135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-57b56d7f-8393-4085-a85f-d21c178b911a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:22.403901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d898360-5c38-4618-b7f0-0c2494b010ce None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:22.444224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e636cc6-975b-48e6-a186-e7633e38e816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:22.445015 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e636cc6-975b-48e6-a186-e7633e38e816 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:22.445371 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e636cc6-975b-48e6-a186-e7633e38e816 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:22.445855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e636cc6-975b-48e6-a186-e7633e38e816 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:22.446696 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e636cc6-975b-48e6-a186-e7633e38e816 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:22.447184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e636cc6-975b-48e6-a186-e7633e38e816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:22.448707 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e636cc6-975b-48e6-a186-e7633e38e816 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:22.458270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 41/81] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:22.459155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e636cc6-975b-48e6-a186-e7633e38e816 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:22.470016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 41/82] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:22.470016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b56d7f-8393-4085-a85f-d21c178b911a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:22.548685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:22.552026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:22.592253 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b56d7f-8393-4085-a85f-d21c178b911a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:22.592494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e636cc6-975b-48e6-a186-e7633e38e816 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:22.631271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82362f70-6d7b-4f2e-b5aa-371669fbff6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:22.632172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82362f70-6d7b-4f2e-b5aa-371669fbff6d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:22.632453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82362f70-6d7b-4f2e-b5aa-371669fbff6d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:22.632729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82362f70-6d7b-4f2e-b5aa-371669fbff6d None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:22.632881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574d39a-fc3e-463a-885b-384f0e149ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:22.634294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574d39a-fc3e-463a-885b-384f0e149ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:22.634699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574d39a-fc3e-463a-885b-384f0e149ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:22.634857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82362f70-6d7b-4f2e-b5aa-371669fbff6d None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:22.635179 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574d39a-fc3e-463a-885b-384f0e149ca7 None admin] PATH_INFO: `/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:22.636297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82362f70-6d7b-4f2e-b5aa-371669fbff6d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:22.636689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574d39a-fc3e-463a-885b-384f0e149ca7 None admin] RBAC: Authorizing `identity:get_user(user_id=6c3cd73e4b2b4c3e83764570cbced9fa)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:22.638150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 42/83] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:22 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:22.638519 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574d39a-fc3e-463a-885b-384f0e149ca7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:22.638965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82362f70-6d7b-4f2e-b5aa-371669fbff6d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:22.641823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 42/84] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:22 2022] GET /identity/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:22.643114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574d39a-fc3e-463a-885b-384f0e149ca7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:22.769554 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82362f70-6d7b-4f2e-b5aa-371669fbff6d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:22.786912 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574d39a-fc3e-463a-885b-384f0e149ca7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:22.815875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2937d-669d-4a73-855b-8c3ffca22d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:22.816776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2937d-669d-4a73-855b-8c3ffca22d25 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:22.817235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2937d-669d-4a73-855b-8c3ffca22d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:22.817722 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2937d-669d-4a73-855b-8c3ffca22d25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:22.835090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d062110-6b8a-44c8-b7b5-3f23d7f33a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:22.836558 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d062110-6b8a-44c8-b7b5-3f23d7f33a92 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:22.836935 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d062110-6b8a-44c8-b7b5-3f23d7f33a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:22.837163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:22.837408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d062110-6b8a-44c8-b7b5-3f23d7f33a92 None admin] PATH_INFO: `/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:22.839920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d062110-6b8a-44c8-b7b5-3f23d7f33a92 None admin] RBAC: Authorizing `identity:get_user(user_id=6c3cd73e4b2b4c3e83764570cbced9fa)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:22.842368 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d062110-6b8a-44c8-b7b5-3f23d7f33a92 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:22.844336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 43/85] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:22 2022] GET /identity/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:22.845338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d062110-6b8a-44c8-b7b5-3f23d7f33a92 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:22.856338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:22.918183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2937d-669d-4a73-855b-8c3ffca22d25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:22.920003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2937d-669d-4a73-855b-8c3ffca22d25 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:23.011657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d062110-6b8a-44c8-b7b5-3f23d7f33a92 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:23.061905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4a920-536f-49e6-94e3-08a71b84606d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:23.062874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4a920-536f-49e6-94e3-08a71b84606d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:23.063244 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4a920-536f-49e6-94e3-08a71b84606d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:23.063610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4a920-536f-49e6-94e3-08a71b84606d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:23.065022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4a920-536f-49e6-94e3-08a71b84606d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:23.067153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4a920-536f-49e6-94e3-08a71b84606d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:23.068957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 44/86] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:23.069890 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4a920-536f-49e6-94e3-08a71b84606d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:23.077886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 43/87] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:23.077886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2937d-669d-4a73-855b-8c3ffca22d25 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:23.143293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:23.181715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:23.218503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4a920-536f-49e6-94e3-08a71b84606d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:23.222871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2937d-669d-4a73-855b-8c3ffca22d25 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:23.258942 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256f357-d782-4ef8-a04e-24f2cb67ede8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:23.260027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256f357-d782-4ef8-a04e-24f2cb67ede8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:23.260493 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256f357-d782-4ef8-a04e-24f2cb67ede8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:23.260947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256f357-d782-4ef8-a04e-24f2cb67ede8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:23.269401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3aa7b-ab6a-4a36-867f-047992833043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:23.270144 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3aa7b-ab6a-4a36-867f-047992833043 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:23.270427 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3aa7b-ab6a-4a36-867f-047992833043 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:23.270705 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3aa7b-ab6a-4a36-867f-047992833043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:23.271562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3aa7b-ab6a-4a36-867f-047992833043 None admin] RBAC: Adding query filter params (role.id=3edce31396af478ea1ea7b76d33da530, scope.project.id=d44ad181a25048dea48323087ad2f3ca, user.id=6c3cd73e4b2b4c3e83764570cbced9fa) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:23.272421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3aa7b-ab6a-4a36-867f-047992833043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:23.274728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3aa7b-ab6a-4a36-867f-047992833043 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:23.281374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 44/88] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:23 2022] GET /identity/v3/role_assignments?user.id=6c3cd73e4b2b4c3e83764570cbced9fa&scope.project.id=d44ad181a25048dea48323087ad2f3ca&role.id=3edce31396af478ea1ea7b76d33da530 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:23.282269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3aa7b-ab6a-4a36-867f-047992833043 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:23.384188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256f357-d782-4ef8-a04e-24f2cb67ede8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:23.386280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256f357-d782-4ef8-a04e-24f2cb67ede8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:23.467251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3aa7b-ab6a-4a36-867f-047992833043 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:23.471979 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:23.509689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:23.529024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4864422-5d75-4cc5-a75a-b5dee540015e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:23.529769 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4864422-5d75-4cc5-a75a-b5dee540015e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:23.530114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4864422-5d75-4cc5-a75a-b5dee540015e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:23.530403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4864422-5d75-4cc5-a75a-b5dee540015e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:23.554348 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 45/89] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:23.556062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256f357-d782-4ef8-a04e-24f2cb67ede8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:23.660557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4864422-5d75-4cc5-a75a-b5dee540015e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:23.662452 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4864422-5d75-4cc5-a75a-b5dee540015e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:23.729597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256f357-d782-4ef8-a04e-24f2cb67ede8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:23.790709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3769f84-247c-47ad-a84e-6a071ee9eff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:23.791792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3769f84-247c-47ad-a84e-6a071ee9eff2 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:23.792246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3769f84-247c-47ad-a84e-6a071ee9eff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:23.792695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3769f84-247c-47ad-a84e-6a071ee9eff2 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:23.793966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3769f84-247c-47ad-a84e-6a071ee9eff2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:23.796306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3769f84-247c-47ad-a84e-6a071ee9eff2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:23.832911 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 46/90] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:49:23 2022] POST /identity/v3/users => generated 289 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:49:23.833817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3769f84-247c-47ad-a84e-6a071ee9eff2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:23.848762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 45/91] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:23.851905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4864422-5d75-4cc5-a75a-b5dee540015e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:23.860768 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:23.877784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:24.000551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4864422-5d75-4cc5-a75a-b5dee540015e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:24.039001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3769f84-247c-47ad-a84e-6a071ee9eff2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:24.045392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06669add-8066-49a0-a05a-db4a07d5b8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:24.046353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06669add-8066-49a0-a05a-db4a07d5b8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:24.046642 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06669add-8066-49a0-a05a-db4a07d5b8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:24.046923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06669add-8066-49a0-a05a-db4a07d5b8c3 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:24.048247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06669add-8066-49a0-a05a-db4a07d5b8c3 None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:24.052336 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06669add-8066-49a0-a05a-db4a07d5b8c3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:24.052336 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 46/92] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:23 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:24.052725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06669add-8066-49a0-a05a-db4a07d5b8c3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:24.101498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109899b-7785-4b65-af22-fc1347684e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:24.102832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109899b-7785-4b65-af22-fc1347684e2d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:24.103127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109899b-7785-4b65-af22-fc1347684e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:24.103471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109899b-7785-4b65-af22-fc1347684e2d None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:24.104923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109899b-7785-4b65-af22-fc1347684e2d None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:24.108835 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109899b-7785-4b65-af22-fc1347684e2d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:24.108835 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 47/93] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:23 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:24.110303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109899b-7785-4b65-af22-fc1347684e2d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:24.177577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:24.217052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06669add-8066-49a0-a05a-db4a07d5b8c3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:24.261514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:24.268459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109899b-7785-4b65-af22-fc1347684e2d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:24.278033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf1aed-f45b-434e-bf61-755590a780d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:24.278984 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf1aed-f45b-434e-bf61-755590a780d4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:24.279268 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf1aed-f45b-434e-bf61-755590a780d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:24.279553 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf1aed-f45b-434e-bf61-755590a780d4 None admin] PATH_INFO: `/v3/projects/3ee53f9ed78b490d934f4ac8566dbcec` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:24.280841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf1aed-f45b-434e-bf61-755590a780d4 None admin] RBAC: Authorizing `identity:get_project(project_id=3ee53f9ed78b490d934f4ac8566dbcec)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:24.282464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf1aed-f45b-434e-bf61-755590a780d4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:24.284505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 47/94] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:24 2022] GET /identity/v3/projects/3ee53f9ed78b490d934f4ac8566dbcec => generated 312 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:24.285102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf1aed-f45b-434e-bf61-755590a780d4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:24.347188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea003d2-1e66-49d3-8d04-2d41dcc0be8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:24.347951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea003d2-1e66-49d3-8d04-2d41dcc0be8a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:24.348238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea003d2-1e66-49d3-8d04-2d41dcc0be8a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:24.348521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea003d2-1e66-49d3-8d04-2d41dcc0be8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:24.349490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea003d2-1e66-49d3-8d04-2d41dcc0be8a None admin] RBAC: Adding query filter params (role.id=794ff5f5032d40a98e67fb0df718b75c, scope.project.id=d44ad181a25048dea48323087ad2f3ca, user.id=6c3cd73e4b2b4c3e83764570cbced9fa) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:24.349754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea003d2-1e66-49d3-8d04-2d41dcc0be8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:24.351426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea003d2-1e66-49d3-8d04-2d41dcc0be8a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:24.358257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 48/95] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:24 2022] GET /identity/v3/role_assignments?user.id=6c3cd73e4b2b4c3e83764570cbced9fa&scope.project.id=d44ad181a25048dea48323087ad2f3ca&role.id=794ff5f5032d40a98e67fb0df718b75c => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:24.359460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea003d2-1e66-49d3-8d04-2d41dcc0be8a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:24.436600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf1aed-f45b-434e-bf61-755590a780d4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:24.486994 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea003d2-1e66-49d3-8d04-2d41dcc0be8a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:24.498611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b2b1f-0a1e-4daa-bab4-287196724f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:24.499317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2b1f-0a1e-4daa-bab4-287196724f54 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:24.499610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2b1f-0a1e-4daa-bab4-287196724f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:24.499890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2b1f-0a1e-4daa-bab4-287196724f54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:24.500774 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b2b1f-0a1e-4daa-bab4-287196724f54 None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=d44ad181a25048dea48323087ad2f3ca, user.id=aec7b27e85c74490a82ad099b2e0743e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:24.501095 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b2b1f-0a1e-4daa-bab4-287196724f54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:24.502731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b2b1f-0a1e-4daa-bab4-287196724f54 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:24.512978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 48/96] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:24 2022] GET /identity/v3/role_assignments?user.id=aec7b27e85c74490a82ad099b2e0743e&scope.project.id=d44ad181a25048dea48323087ad2f3ca&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:24.546627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3467c8-cf4d-4c22-baa2-db5186f70a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:24.547332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3467c8-cf4d-4c22-baa2-db5186f70a84 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:24.547682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3467c8-cf4d-4c22-baa2-db5186f70a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:24.547969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3467c8-cf4d-4c22-baa2-db5186f70a84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:24.548903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3467c8-cf4d-4c22-baa2-db5186f70a84 None admin] RBAC: Adding query filter params (role.id=3edce31396af478ea1ea7b76d33da530, scope.project.id=3ee53f9ed78b490d934f4ac8566dbcec, user.id=6c3cd73e4b2b4c3e83764570cbced9fa) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:24.549231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3467c8-cf4d-4c22-baa2-db5186f70a84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:24.550825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3467c8-cf4d-4c22-baa2-db5186f70a84 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:24.558354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 49/97] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:24 2022] GET /identity/v3/role_assignments?user.id=6c3cd73e4b2b4c3e83764570cbced9fa&scope.project.id=3ee53f9ed78b490d934f4ac8566dbcec&role.id=3edce31396af478ea1ea7b76d33da530 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:26.232880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83023cc2-5d20-4fa7-8ad6-f12479113263 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:26.232880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83023cc2-5d20-4fa7-8ad6-f12479113263 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:26.232880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83023cc2-5d20-4fa7-8ad6-f12479113263 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:26.232880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 49/98] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:26.239721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c4517-54a2-4956-8ca2-71f4eb170290 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:26.240698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c4517-54a2-4956-8ca2-71f4eb170290 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:26.241192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c4517-54a2-4956-8ca2-71f4eb170290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:26.260753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ca7c4517-54a2-4956-8ca2-71f4eb170290 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:26.262426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ca7c4517-54a2-4956-8ca2-71f4eb170290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:26.287010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:26.344055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 50/99] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:26.368959 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83023cc2-5d20-4fa7-8ad6-f12479113263 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:26.373515 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:26.541957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83023cc2-5d20-4fa7-8ad6-f12479113263 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:26.595299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350da43b-f410-4840-a243-d7fbe481cf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:26.596540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350da43b-f410-4840-a243-d7fbe481cf62 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:26.597049 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350da43b-f410-4840-a243-d7fbe481cf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:26.597530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350da43b-f410-4840-a243-d7fbe481cf62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:26.618932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350da43b-f410-4840-a243-d7fbe481cf62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-350da43b-f410-4840-a243-d7fbe481cf62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:26.627411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:26.628339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:26.629501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:26.629501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:26.629501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:26.629501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:26.629501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:26.629501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:26.629501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:26.629501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:26.629501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:26.629501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:26.629501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:26.629501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:26.636018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 50/100] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:26.642056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c4517-54a2-4956-8ca2-71f4eb170290 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:26.768825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c4517-54a2-4956-8ca2-71f4eb170290 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:26.809400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfbcc5-7832-42cc-8d8f-3902d5a75743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:26.810164 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfbcc5-7832-42cc-8d8f-3902d5a75743 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:26.810453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfbcc5-7832-42cc-8d8f-3902d5a75743 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:26.810740 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfbcc5-7832-42cc-8d8f-3902d5a75743 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:26.811549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfbcc5-7832-42cc-8d8f-3902d5a75743 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:26.811862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfbcc5-7832-42cc-8d8f-3902d5a75743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:26.813318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfbcc5-7832-42cc-8d8f-3902d5a75743 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:26.823964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 51/101] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:26.830349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350da43b-f410-4840-a243-d7fbe481cf62 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:26.941738 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:26.993675 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350da43b-f410-4840-a243-d7fbe481cf62 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:27.044980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66950c-15ae-4f71-808f-11b07c6ec9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:27.046353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66950c-15ae-4f71-808f-11b07c6ec9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:27.046848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66950c-15ae-4f71-808f-11b07c6ec9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:27.047321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66950c-15ae-4f71-808f-11b07c6ec9d1 None admin] PATH_INFO: `/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:27.054604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66950c-15ae-4f71-808f-11b07c6ec9d1 None admin] RBAC: Authorizing `identity:get_user(user_id=6c3cd73e4b2b4c3e83764570cbced9fa)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:27.056573 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66950c-15ae-4f71-808f-11b07c6ec9d1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:27.059047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 51/102] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:26 2022] GET /identity/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:27.065306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfbcc5-7832-42cc-8d8f-3902d5a75743 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:27.119183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:27.266115 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfbcc5-7832-42cc-8d8f-3902d5a75743 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:27.306222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281d698-8e1b-4006-8b17-fe96151a95e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:27.307129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281d698-8e1b-4006-8b17-fe96151a95e5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:27.307421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281d698-8e1b-4006-8b17-fe96151a95e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:27.307707 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281d698-8e1b-4006-8b17-fe96151a95e5 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:27.309013 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281d698-8e1b-4006-8b17-fe96151a95e5 None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:27.310585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281d698-8e1b-4006-8b17-fe96151a95e5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:27.312564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 52/103] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:27 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:27.322790 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66950c-15ae-4f71-808f-11b07c6ec9d1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:27.465959 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66950c-15ae-4f71-808f-11b07c6ec9d1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:27.506067 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:27.516876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd9ff6-16c7-459f-a5fc-0868b8367bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:27.517739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd9ff6-16c7-459f-a5fc-0868b8367bef None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:27.518027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd9ff6-16c7-459f-a5fc-0868b8367bef None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:27.518305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd9ff6-16c7-459f-a5fc-0868b8367bef None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca/users/6c3cd73e4b2b4c3e83764570cbced9fa/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:27.520387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd9ff6-16c7-459f-a5fc-0868b8367bef None admin] RBAC: Authorizing `identity:create_grant(project_id=d44ad181a25048dea48323087ad2f3ca, user_id=6c3cd73e4b2b4c3e83764570cbced9fa, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:27.521896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd9ff6-16c7-459f-a5fc-0868b8367bef None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:27.548475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 52/104] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:49:27 2022] PUT /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca/users/6c3cd73e4b2b4c3e83764570cbced9fa/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:49:27.730356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18532524-bd64-458f-8ea0-e287b606fd84 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:27.731073 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18532524-bd64-458f-8ea0-e287b606fd84 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:27.731579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18532524-bd64-458f-8ea0-e287b606fd84 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:27.733306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 53/105] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:49:27.739245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06688e3d-c069-4652-9a04-11ebf22f295b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:27.739780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06688e3d-c069-4652-9a04-11ebf22f295b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:27.740221 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06688e3d-c069-4652-9a04-11ebf22f295b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:27.766755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-06688e3d-c069-4652-9a04-11ebf22f295b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:27.768319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-06688e3d-c069-4652-9a04-11ebf22f295b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:27.867226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 53/106] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:27.885581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76e5bd-c6bb-401b-8100-2d5baf55b034 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:27.886205 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76e5bd-c6bb-401b-8100-2d5baf55b034 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:27.886627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76e5bd-c6bb-401b-8100-2d5baf55b034 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:27.888502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 54/107] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:27.894124 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6863a-65e2-4d13-af54-317f6542a687 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:27.894633 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6863a-65e2-4d13-af54-317f6542a687 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:27.895131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6863a-65e2-4d13-af54-317f6542a687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:27.901019 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76e5bd-c6bb-401b-8100-2d5baf55b034 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:27.922860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-46c6863a-65e2-4d13-af54-317f6542a687 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:27.924378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-46c6863a-65e2-4d13-af54-317f6542a687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:28.046770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 54/108] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:28.070488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:28.072980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6863a-65e2-4d13-af54-317f6542a687 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:28.115397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76e5bd-c6bb-401b-8100-2d5baf55b034 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:28.182081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:28.222762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6863a-65e2-4d13-af54-317f6542a687 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:28.223039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df0561-515d-4104-b71a-d4d3501fec74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:28.224431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df0561-515d-4104-b71a-d4d3501fec74 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:28.224910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df0561-515d-4104-b71a-d4d3501fec74 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:28.225379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df0561-515d-4104-b71a-d4d3501fec74 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:28.262042 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094a0bb-64e2-4a34-ac1f-41c61d97ac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:28.263178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094a0bb-64e2-4a34-ac1f-41c61d97ac56 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:28.263608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094a0bb-64e2-4a34-ac1f-41c61d97ac56 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:28.264027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094a0bb-64e2-4a34-ac1f-41c61d97ac56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:28.270772 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df0561-515d-4104-b71a-d4d3501fec74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:28.276936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094a0bb-64e2-4a34-ac1f-41c61d97ac56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7094a0bb-64e2-4a34-ac1f-41c61d97ac56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:28.285590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:28.286780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:28.287770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:28.287770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:28.287770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:28.287770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:28.287770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:28.287770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:28.287770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:28.287770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:28.287770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:28.287770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:28.287770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:49:28.287770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e6df0561-515d-4104-b71a-d4d3501fec74 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:28.288518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:28.289405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:28.290903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 55/109] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:28.293723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 55/110] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:49:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:49:28.299820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df0561-515d-4104-b71a-d4d3501fec74 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:28.301122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094a0bb-64e2-4a34-ac1f-41c61d97ac56 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:28.464857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094a0bb-64e2-4a34-ac1f-41c61d97ac56 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:28.483258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:28.515084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a706d4-7449-4a3d-abfb-1dbf402eacfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:28.515881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a706d4-7449-4a3d-abfb-1dbf402eacfd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:28.516166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a706d4-7449-4a3d-abfb-1dbf402eacfd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:28.516443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a706d4-7449-4a3d-abfb-1dbf402eacfd None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:28.517313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a706d4-7449-4a3d-abfb-1dbf402eacfd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:28.517652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a706d4-7449-4a3d-abfb-1dbf402eacfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:28.519609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a706d4-7449-4a3d-abfb-1dbf402eacfd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:28.531215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df0561-515d-4104-b71a-d4d3501fec74 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:28.536269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 56/111] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:28.536269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ef5b9-5e20-4620-9a6c-231caca9a3c2 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:28.536269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ef5b9-5e20-4620-9a6c-231caca9a3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:28.536269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ef5b9-5e20-4620-9a6c-231caca9a3c2 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:28.536269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 57/112] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:28.537667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a1145-a8d1-4a6b-adf9-421353497a6c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:28.537829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a1145-a8d1-4a6b-adf9-421353497a6c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:28.538808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a1145-a8d1-4a6b-adf9-421353497a6c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:28.540618 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 58/113] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:28.543782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856af3b-59e0-4988-83fb-338407241a1c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:28.544553 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856af3b-59e0-4988-83fb-338407241a1c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:28.545302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856af3b-59e0-4988-83fb-338407241a1c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:28.547982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 59/114] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:28.550528 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dd7bf-b6dc-4d80-82a4-1aedc1c837f9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:28.551242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dd7bf-b6dc-4d80-82a4-1aedc1c837f9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:28.552767 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dd7bf-b6dc-4d80-82a4-1aedc1c837f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:28.575566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3d9dd7bf-b6dc-4d80-82a4-1aedc1c837f9 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:28.576793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3d9dd7bf-b6dc-4d80-82a4-1aedc1c837f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:28.595057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01f672-4a27-4641-8f5f-c76ba829f12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:28.595825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01f672-4a27-4641-8f5f-c76ba829f12d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:28.596026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01f672-4a27-4641-8f5f-c76ba829f12d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:28.596306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01f672-4a27-4641-8f5f-c76ba829f12d None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:28.597072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01f672-4a27-4641-8f5f-c76ba829f12d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:28.597380 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01f672-4a27-4641-8f5f-c76ba829f12d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:28.598854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01f672-4a27-4641-8f5f-c76ba829f12d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:28.608841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 56/115] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:49:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:28.609573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01f672-4a27-4641-8f5f-c76ba829f12d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:28.623117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:28.672789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 60/116] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:28.676004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b119145-13de-4936-90cf-2fc6c6098bb3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:28.676276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b119145-13de-4936-90cf-2fc6c6098bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:28.676725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b119145-13de-4936-90cf-2fc6c6098bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:28.706958 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7b119145-13de-4936-90cf-2fc6c6098bb3 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:28.709219 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7b119145-13de-4936-90cf-2fc6c6098bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:28.784813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01f672-4a27-4641-8f5f-c76ba829f12d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:28.813813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 61/117] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:28.813813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f9589-b907-41f7-bffb-bb5d3adaae41 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:28.813813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f9589-b907-41f7-bffb-bb5d3adaae41 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:28.813813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f9589-b907-41f7-bffb-bb5d3adaae41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:28.834841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc1eaa-73d4-453c-a5ee-a4e98dc9f612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:28.835753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc1eaa-73d4-453c-a5ee-a4e98dc9f612 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:28.836038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc1eaa-73d4-453c-a5ee-a4e98dc9f612 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:28.836441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc1eaa-73d4-453c-a5ee-a4e98dc9f612 None admin] PATH_INFO: `/v3/users/4d814c6361c14ec08243293cf4dec26a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:28.841750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-af3f9589-b907-41f7-bffb-bb5d3adaae41 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:28.843321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-af3f9589-b907-41f7-bffb-bb5d3adaae41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:28.855730 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc1eaa-73d4-453c-a5ee-a4e98dc9f612 None admin] RBAC: Authorizing `identity:get_user(user_id=4d814c6361c14ec08243293cf4dec26a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:28.857348 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc1eaa-73d4-453c-a5ee-a4e98dc9f612 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:28.859592 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 57/118] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:28 2022] GET /identity/v3/users/4d814c6361c14ec08243293cf4dec26a => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:28.864441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc1eaa-73d4-453c-a5ee-a4e98dc9f612 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:28.900539 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:28.957450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:28.959001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 62/119] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:28.960122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f9589-b907-41f7-bffb-bb5d3adaae41 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:29.018851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc1eaa-73d4-453c-a5ee-a4e98dc9f612 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:29.064216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367412b9-610d-43f5-a80d-70a0abc49f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:29.065525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367412b9-610d-43f5-a80d-70a0abc49f5f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:29.066028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367412b9-610d-43f5-a80d-70a0abc49f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:29.066451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367412b9-610d-43f5-a80d-70a0abc49f5f None admin] PATH_INFO: `/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:29.068066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367412b9-610d-43f5-a80d-70a0abc49f5f None admin] RBAC: Authorizing `identity:get_user(user_id=6c3cd73e4b2b4c3e83764570cbced9fa)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:29.070927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367412b9-610d-43f5-a80d-70a0abc49f5f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:29.079275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 58/120] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:28 2022] GET /identity/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:29.079275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367412b9-610d-43f5-a80d-70a0abc49f5f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:29.122950 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f9589-b907-41f7-bffb-bb5d3adaae41 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:29.179592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3486a-3f11-4ba2-900c-fd0dd074bde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:29.181084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3486a-3f11-4ba2-900c-fd0dd074bde7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:29.181645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3486a-3f11-4ba2-900c-fd0dd074bde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:29.182163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3486a-3f11-4ba2-900c-fd0dd074bde7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:29.199593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3486a-3f11-4ba2-900c-fd0dd074bde7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-96a3486a-3f11-4ba2-900c-fd0dd074bde7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:29.213851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:29.215166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 63/121] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:29.216075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3486a-3f11-4ba2-900c-fd0dd074bde7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:29.236725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367412b9-610d-43f5-a80d-70a0abc49f5f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:29.261922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:29.293200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71dcfb6-86ed-49eb-94dd-5958abc384af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:29.294666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71dcfb6-86ed-49eb-94dd-5958abc384af None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:29.295170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71dcfb6-86ed-49eb-94dd-5958abc384af None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:29.295815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71dcfb6-86ed-49eb-94dd-5958abc384af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:29.298131 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71dcfb6-86ed-49eb-94dd-5958abc384af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:29.300823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71dcfb6-86ed-49eb-94dd-5958abc384af None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:29.309382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 59/122] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:29.309382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71dcfb6-86ed-49eb-94dd-5958abc384af None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:29.312820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:29.401047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3486a-3f11-4ba2-900c-fd0dd074bde7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:29.469223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfa74e-968c-4577-a2c8-04ad98a7c9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:29.470318 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfa74e-968c-4577-a2c8-04ad98a7c9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:29.470825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfa74e-968c-4577-a2c8-04ad98a7c9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:29.471334 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfa74e-968c-4577-a2c8-04ad98a7c9b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:29.490849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71dcfb6-86ed-49eb-94dd-5958abc384af None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:29.555361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b3d04-4ecb-480e-b807-f1991a430d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:29.556586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b3d04-4ecb-480e-b807-f1991a430d75 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:29.557066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b3d04-4ecb-480e-b807-f1991a430d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:29.557533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b3d04-4ecb-480e-b807-f1991a430d75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:29.562773 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfa74e-968c-4577-a2c8-04ad98a7c9b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:29.564329 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfa74e-968c-4577-a2c8-04ad98a7c9b7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:29.577237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b3d04-4ecb-480e-b807-f1991a430d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4a5b3d04-4ecb-480e-b807-f1991a430d75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:29.594504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:29.595587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:29.596506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:29.596506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:29.596506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:29.596506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:29.596506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:29.596506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:29.596506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:29.596506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:29.596506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:29.596506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:29.596506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:29.596506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:29.597183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 60/123] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:49:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:29.603082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b3d04-4ecb-480e-b807-f1991a430d75 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:29.610668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:29.684858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:29.731739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 64/124] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:29.731739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfa74e-968c-4577-a2c8-04ad98a7c9b7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:29.741096 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b3d04-4ecb-480e-b807-f1991a430d75 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:29.783213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f656b1-d3ac-45f9-937c-01ebf09e6728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:29.784220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f656b1-d3ac-45f9-937c-01ebf09e6728 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:29.784571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f656b1-d3ac-45f9-937c-01ebf09e6728 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:29.784955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f656b1-d3ac-45f9-937c-01ebf09e6728 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:29.786370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f656b1-d3ac-45f9-937c-01ebf09e6728 None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:29.788006 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f656b1-d3ac-45f9-937c-01ebf09e6728 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:29.790208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 61/125] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:29 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:29.791038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f656b1-d3ac-45f9-937c-01ebf09e6728 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:29.870274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfa74e-968c-4577-a2c8-04ad98a7c9b7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:29.906735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:29.921431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8c18d-ca71-4bd3-8f6c-b20b4a26452e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:29.922305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8c18d-ca71-4bd3-8f6c-b20b4a26452e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:29.922724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8c18d-ca71-4bd3-8f6c-b20b4a26452e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:29.923106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8c18d-ca71-4bd3-8f6c-b20b4a26452e None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:29.923987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8c18d-ca71-4bd3-8f6c-b20b4a26452e None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:29.924398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8c18d-ca71-4bd3-8f6c-b20b4a26452e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:29.925953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8c18d-ca71-4bd3-8f6c-b20b4a26452e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:29.942081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 65/126] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:29.942257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f656b1-d3ac-45f9-937c-01ebf09e6728 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:29.942351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8c18d-ca71-4bd3-8f6c-b20b4a26452e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:29.983689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:29.993691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c381f7e-5e75-44c0-802f-f43a8cdf4c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:29.994830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c381f7e-5e75-44c0-802f-f43a8cdf4c14 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:29.995324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c381f7e-5e75-44c0-802f-f43a8cdf4c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:29.995846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c381f7e-5e75-44c0-802f-f43a8cdf4c14 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:29.997235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c381f7e-5e75-44c0-802f-f43a8cdf4c14 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:29.999800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c381f7e-5e75-44c0-802f-f43a8cdf4c14 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:30.044107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 62/127] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:49:29 2022] POST /identity/v3/projects => generated 302 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:49:30.045595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c381f7e-5e75-44c0-802f-f43a8cdf4c14 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:30.095981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8c18d-ca71-4bd3-8f6c-b20b4a26452e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:30.162195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa228a7-64cd-4187-baaf-94fad15d29e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:30.163016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa228a7-64cd-4187-baaf-94fad15d29e9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:30.163376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa228a7-64cd-4187-baaf-94fad15d29e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:30.163756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa228a7-64cd-4187-baaf-94fad15d29e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:30.166028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa228a7-64cd-4187-baaf-94fad15d29e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:30.166426 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa228a7-64cd-4187-baaf-94fad15d29e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:30.167938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa228a7-64cd-4187-baaf-94fad15d29e9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:30.180045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 66/128] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:30.181501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa228a7-64cd-4187-baaf-94fad15d29e9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:30.216495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c381f7e-5e75-44c0-802f-f43a8cdf4c14 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:30.247163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:30.281932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d58bd6-a505-4cf4-8167-85544297f2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:30.282832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d58bd6-a505-4cf4-8167-85544297f2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:30.283113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d58bd6-a505-4cf4-8167-85544297f2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:30.283387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d58bd6-a505-4cf4-8167-85544297f2a2 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:30.284704 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d58bd6-a505-4cf4-8167-85544297f2a2 None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:30.286238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d58bd6-a505-4cf4-8167-85544297f2a2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:30.288034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 63/129] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:30 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:30.288841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d58bd6-a505-4cf4-8167-85544297f2a2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:30.329701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa228a7-64cd-4187-baaf-94fad15d29e9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:30.389298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:30.394403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b61d4-2439-4899-bd40-2ae8bd1cb119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:30.395464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b61d4-2439-4899-bd40-2ae8bd1cb119 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:30.395805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b61d4-2439-4899-bd40-2ae8bd1cb119 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:30.396139 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b61d4-2439-4899-bd40-2ae8bd1cb119 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca/users/6c3cd73e4b2b4c3e83764570cbced9fa/roles/794ff5f5032d40a98e67fb0df718b75c` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:30.400143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b61d4-2439-4899-bd40-2ae8bd1cb119 None admin] RBAC: Authorizing `identity:create_grant(project_id=d44ad181a25048dea48323087ad2f3ca, user_id=6c3cd73e4b2b4c3e83764570cbced9fa, role_id=794ff5f5032d40a98e67fb0df718b75c)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:30.402406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b61d4-2439-4899-bd40-2ae8bd1cb119 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:30.422454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 67/130] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:49:30 2022] PUT /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca/users/6c3cd73e4b2b4c3e83764570cbced9fa/roles/794ff5f5032d40a98e67fb0df718b75c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:49:30.430805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b61d4-2439-4899-bd40-2ae8bd1cb119 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:30.475687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d58bd6-a505-4cf4-8167-85544297f2a2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:30.562123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e32e89-2407-452b-a291-5db55ed79eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:30.563427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e32e89-2407-452b-a291-5db55ed79eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:30.563793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e32e89-2407-452b-a291-5db55ed79eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:30.564300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e32e89-2407-452b-a291-5db55ed79eb5 None admin] PATH_INFO: `/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:30.565907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e32e89-2407-452b-a291-5db55ed79eb5 None admin] RBAC: Authorizing `identity:get_user(user_id=6c3cd73e4b2b4c3e83764570cbced9fa)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:30.567658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e32e89-2407-452b-a291-5db55ed79eb5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:30.570327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 64/131] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:30 2022] GET /identity/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa => generated 275 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:30.571934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc31b1-cbe8-427f-b1da-07a3b5b0b355 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:30.572233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc31b1-cbe8-427f-b1da-07a3b5b0b355 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:30.572591 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc31b1-cbe8-427f-b1da-07a3b5b0b355 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:30.574271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 65/132] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:30.574902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc31b1-cbe8-427f-b1da-07a3b5b0b355 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:30.605239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b61d4-2439-4899-bd40-2ae8bd1cb119 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:30.621711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:30.665327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168016cd-42f8-4e90-936c-104e2129691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:30.666502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168016cd-42f8-4e90-936c-104e2129691b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:30.666861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168016cd-42f8-4e90-936c-104e2129691b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:30.667236 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168016cd-42f8-4e90-936c-104e2129691b None admin] PATH_INFO: `/v3/users/aec7b27e85c74490a82ad099b2e0743e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:30.668599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168016cd-42f8-4e90-936c-104e2129691b None admin] RBAC: Authorizing `identity:get_user(user_id=aec7b27e85c74490a82ad099b2e0743e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:30.670303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168016cd-42f8-4e90-936c-104e2129691b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:30.672216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 68/133] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:30 2022] GET /identity/v3/users/aec7b27e85c74490a82ad099b2e0743e => generated 247 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:30.677074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168016cd-42f8-4e90-936c-104e2129691b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:30.730259 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc31b1-cbe8-427f-b1da-07a3b5b0b355 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:30.753340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:30.785250 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd58d0f-9cc8-4d78-9d78-67b1dfdb0a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:30.786430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd58d0f-9cc8-4d78-9d78-67b1dfdb0a70 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:30.786975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd58d0f-9cc8-4d78-9d78-67b1dfdb0a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:30.787414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd58d0f-9cc8-4d78-9d78-67b1dfdb0a70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:30.788872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd58d0f-9cc8-4d78-9d78-67b1dfdb0a70 None admin] RBAC: Adding query filter params (role.id=48f409e647e64284817bad2b8ec30bf2, scope.project.id=d44ad181a25048dea48323087ad2f3ca, user.id=4d814c6361c14ec08243293cf4dec26a) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:30.789511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd58d0f-9cc8-4d78-9d78-67b1dfdb0a70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:30.791493 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd58d0f-9cc8-4d78-9d78-67b1dfdb0a70 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:30.800170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 66/134] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:30 2022] GET /identity/v3/role_assignments?user.id=4d814c6361c14ec08243293cf4dec26a&scope.project.id=d44ad181a25048dea48323087ad2f3ca&role.id=48f409e647e64284817bad2b8ec30bf2 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:30.801601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12b233-923b-4c49-8643-53046fb3d94c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:30.810314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12b233-923b-4c49-8643-53046fb3d94c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:30.810691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12b233-923b-4c49-8643-53046fb3d94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:30.833313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168016cd-42f8-4e90-936c-104e2129691b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:30.844768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ce12b233-923b-4c49-8643-53046fb3d94c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:30.846124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ce12b233-923b-4c49-8643-53046fb3d94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:30.934507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322198cb-a3a1-4768-9729-e060e7bbedfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:30.941050 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322198cb-a3a1-4768-9729-e060e7bbedfd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:30.942330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322198cb-a3a1-4768-9729-e060e7bbedfd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:30.943302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322198cb-a3a1-4768-9729-e060e7bbedfd None admin] PATH_INFO: `/v3/projects/3ee53f9ed78b490d934f4ac8566dbcec` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:30.950033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322198cb-a3a1-4768-9729-e060e7bbedfd None admin] RBAC: Authorizing `identity:get_project(project_id=3ee53f9ed78b490d934f4ac8566dbcec)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:30.953587 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322198cb-a3a1-4768-9729-e060e7bbedfd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:30.958087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 69/135] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:30 2022] GET /identity/v3/projects/3ee53f9ed78b490d934f4ac8566dbcec => generated 312 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:30.972668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322198cb-a3a1-4768-9729-e060e7bbedfd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:31.042912 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 67/136] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:31.043718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12b233-923b-4c49-8643-53046fb3d94c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:31.086051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:31.207854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322198cb-a3a1-4768-9729-e060e7bbedfd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:31.265705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12b233-923b-4c49-8643-53046fb3d94c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:31.289077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f6541-300e-4c44-bfae-36396d0998e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:31.290316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f6541-300e-4c44-bfae-36396d0998e8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:31.290596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f6541-300e-4c44-bfae-36396d0998e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:31.290881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f6541-300e-4c44-bfae-36396d0998e8 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:31.293269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f6541-300e-4c44-bfae-36396d0998e8 None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:31.294723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f6541-300e-4c44-bfae-36396d0998e8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:31.296806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 70/137] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:30 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:31.300982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f6541-300e-4c44-bfae-36396d0998e8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:31.306035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:31.348253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06316691-a67d-4c12-ad59-8e59618534d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:31.349184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06316691-a67d-4c12-ad59-8e59618534d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:31.349475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06316691-a67d-4c12-ad59-8e59618534d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:31.349792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06316691-a67d-4c12-ad59-8e59618534d4 None admin] PATH_INFO: `/v3/projects/3ee53f9ed78b490d934f4ac8566dbcec/users/6c3cd73e4b2b4c3e83764570cbced9fa/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:31.353700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06316691-a67d-4c12-ad59-8e59618534d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ee53f9ed78b490d934f4ac8566dbcec, user_id=6c3cd73e4b2b4c3e83764570cbced9fa, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:31.356733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06316691-a67d-4c12-ad59-8e59618534d4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:31.386722 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 68/138] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:49:31 2022] PUT /identity/v3/projects/3ee53f9ed78b490d934f4ac8566dbcec/users/6c3cd73e4b2b4c3e83764570cbced9fa/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:49:31.387728 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06316691-a67d-4c12-ad59-8e59618534d4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:31.494311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f6541-300e-4c44-bfae-36396d0998e8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:31.545431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:31.584831 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b05853-88a3-4ccc-89fa-d1d45f6aeddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:31.585752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b05853-88a3-4ccc-89fa-d1d45f6aeddd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:31.586060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b05853-88a3-4ccc-89fa-d1d45f6aeddd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:31.586394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b05853-88a3-4ccc-89fa-d1d45f6aeddd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:31.608404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b05853-88a3-4ccc-89fa-d1d45f6aeddd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c3b05853-88a3-4ccc-89fa-d1d45f6aeddd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:31.625003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:31.626754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:31.628228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 71/139] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:31.634234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b05853-88a3-4ccc-89fa-d1d45f6aeddd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:31.676736 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06316691-a67d-4c12-ad59-8e59618534d4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:31.771137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb6799-8e65-41f2-86bf-9c06cc14528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:31.772168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb6799-8e65-41f2-86bf-9c06cc14528d None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:31.772465 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb6799-8e65-41f2-86bf-9c06cc14528d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:31.772753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb6799-8e65-41f2-86bf-9c06cc14528d None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca/users/aec7b27e85c74490a82ad099b2e0743e/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:31.782662 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb6799-8e65-41f2-86bf-9c06cc14528d None admin] RBAC: Authorizing `identity:create_grant(project_id=d44ad181a25048dea48323087ad2f3ca, user_id=aec7b27e85c74490a82ad099b2e0743e, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:31.784224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb6799-8e65-41f2-86bf-9c06cc14528d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:31.810296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 69/140] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:49:31 2022] PUT /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca/users/aec7b27e85c74490a82ad099b2e0743e/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:49:31.831789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b05853-88a3-4ccc-89fa-d1d45f6aeddd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:31.910447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820c747-32b6-41f2-bf43-1f271ef78ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:31.911196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820c747-32b6-41f2-bf43-1f271ef78ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:31.911493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820c747-32b6-41f2-bf43-1f271ef78ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:31.911785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820c747-32b6-41f2-bf43-1f271ef78ec0 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:31.912586 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820c747-32b6-41f2-bf43-1f271ef78ec0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:31.912919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820c747-32b6-41f2-bf43-1f271ef78ec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:31.914509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820c747-32b6-41f2-bf43-1f271ef78ec0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:31.925817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 72/141] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:31.932031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb6799-8e65-41f2-86bf-9c06cc14528d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:32.006085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:32.117655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb6799-8e65-41f2-86bf-9c06cc14528d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:32.162785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f1df9-16fb-4a40-bfba-3e996aefe110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:32.163766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f1df9-16fb-4a40-bfba-3e996aefe110 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:32.164063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f1df9-16fb-4a40-bfba-3e996aefe110 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:32.164476 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f1df9-16fb-4a40-bfba-3e996aefe110 None admin] PATH_INFO: `/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:32.166490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f1df9-16fb-4a40-bfba-3e996aefe110 None admin] RBAC: Authorizing `identity:get_user(user_id=6c3cd73e4b2b4c3e83764570cbced9fa)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:32.168050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f1df9-16fb-4a40-bfba-3e996aefe110 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:32.169994 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 70/142] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:31 2022] GET /identity/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:32.176091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820c747-32b6-41f2-bf43-1f271ef78ec0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:32.224746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:32.306820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820c747-32b6-41f2-bf43-1f271ef78ec0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:32.387398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7500fd-0e8c-4dcb-96f4-d076690a6146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:32.388995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7500fd-0e8c-4dcb-96f4-d076690a6146 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:32.389765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7500fd-0e8c-4dcb-96f4-d076690a6146 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:32.390534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7500fd-0e8c-4dcb-96f4-d076690a6146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:32.526436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7500fd-0e8c-4dcb-96f4-d076690a6146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:32.529541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7500fd-0e8c-4dcb-96f4-d076690a6146 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:32.605660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:32.754340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 73/143] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:32.760236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f1df9-16fb-4a40-bfba-3e996aefe110 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:32.928211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:32.932094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f1df9-16fb-4a40-bfba-3e996aefe110 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:32.976302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a6b48-5cf2-43a5-a895-cef9147f5586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:32.977402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a6b48-5cf2-43a5-a895-cef9147f5586 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:32.977944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a6b48-5cf2-43a5-a895-cef9147f5586 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:32.978294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a6b48-5cf2-43a5-a895-cef9147f5586 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:32.979636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a6b48-5cf2-43a5-a895-cef9147f5586 None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:32.981140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a6b48-5cf2-43a5-a895-cef9147f5586 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:32.983300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 71/144] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:32 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:32.990172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7500fd-0e8c-4dcb-96f4-d076690a6146 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:33.184626 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677696a-2e3e-48e4-ac17-0736f2bfcd68 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:33.185059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677696a-2e3e-48e4-ac17-0736f2bfcd68 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:33.185241 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677696a-2e3e-48e4-ac17-0736f2bfcd68 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:33.186898 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 72/145] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:33.195007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cf4b0-d04e-49aa-9532-33f3afcad5a8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:33.195386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cf4b0-d04e-49aa-9532-33f3afcad5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:33.196096 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cf4b0-d04e-49aa-9532-33f3afcad5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:33.236316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-269cf4b0-d04e-49aa-9532-33f3afcad5a8 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:33.237495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-269cf4b0-d04e-49aa-9532-33f3afcad5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:33.300575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7500fd-0e8c-4dcb-96f4-d076690a6146 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:33.360524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 73/146] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:33.386562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cf4b0-d04e-49aa-9532-33f3afcad5a8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:33.399548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748809eb-02d6-4574-a283-4f6167961bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:33.400240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748809eb-02d6-4574-a283-4f6167961bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:33.400528 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748809eb-02d6-4574-a283-4f6167961bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:33.400808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748809eb-02d6-4574-a283-4f6167961bc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:33.401702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748809eb-02d6-4574-a283-4f6167961bc2 None admin] RBAC: Adding query filter params (role.id=3edce31396af478ea1ea7b76d33da530, scope.project.id=d44ad181a25048dea48323087ad2f3ca, user.id=6c3cd73e4b2b4c3e83764570cbced9fa) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:33.402018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748809eb-02d6-4574-a283-4f6167961bc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:33.403535 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748809eb-02d6-4574-a283-4f6167961bc2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:33.405659 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:33.420335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 74/147] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:32 2022] GET /identity/v3/role_assignments?user.id=6c3cd73e4b2b4c3e83764570cbced9fa&scope.project.id=d44ad181a25048dea48323087ad2f3ca&role.id=3edce31396af478ea1ea7b76d33da530 => generated 608 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:33.504169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:33.514102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cf4b0-d04e-49aa-9532-33f3afcad5a8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:33.540368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365db1-f28e-4bb3-8bc1-ef3301f99dcf None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:33.541211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365db1-f28e-4bb3-8bc1-ef3301f99dcf None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:33.541746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365db1-f28e-4bb3-8bc1-ef3301f99dcf None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:33.543767 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 75/148] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:33.550323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5520d7-e59e-485d-9fc0-94b160b8c131 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:33.550992 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5520d7-e59e-485d-9fc0-94b160b8c131 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:33.551504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5520d7-e59e-485d-9fc0-94b160b8c131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:33.574275 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3d5520d7-e59e-485d-9fc0-94b160b8c131 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:33.576639 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3d5520d7-e59e-485d-9fc0-94b160b8c131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:33.577711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb71d44-8fd5-4551-b518-e2ae66f94f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:33.578868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb71d44-8fd5-4551-b518-e2ae66f94f7e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:33.579375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb71d44-8fd5-4551-b518-e2ae66f94f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:33.579856 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb71d44-8fd5-4551-b518-e2ae66f94f7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:33.583886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb71d44-8fd5-4551-b518-e2ae66f94f7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:33.585895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb71d44-8fd5-4551-b518-e2ae66f94f7e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:33.588664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 74/149] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:33.596717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb71d44-8fd5-4551-b518-e2ae66f94f7e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:33.728351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 76/150] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:33.784741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5520d7-e59e-485d-9fc0-94b160b8c131 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:33.785401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb71d44-8fd5-4551-b518-e2ae66f94f7e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:33.843855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506cb86-de97-4493-bf9f-bc0e8dbedc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:33.844893 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506cb86-de97-4493-bf9f-bc0e8dbedc6e None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:33.845188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506cb86-de97-4493-bf9f-bc0e8dbedc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:33.845474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506cb86-de97-4493-bf9f-bc0e8dbedc6e None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:33.846277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506cb86-de97-4493-bf9f-bc0e8dbedc6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:33.847805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506cb86-de97-4493-bf9f-bc0e8dbedc6e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:33.891827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 75/151] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:49:33 2022] POST /identity/v3/users => generated 283 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:49:33.977480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5520d7-e59e-485d-9fc0-94b160b8c131 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:34.016470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:34.031291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce19aab-1305-4e0d-86ec-cc4578940bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:34.032184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce19aab-1305-4e0d-86ec-cc4578940bca None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:34.032478 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce19aab-1305-4e0d-86ec-cc4578940bca None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:34.032757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce19aab-1305-4e0d-86ec-cc4578940bca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:34.054807 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce19aab-1305-4e0d-86ec-cc4578940bca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9ce19aab-1305-4e0d-86ec-cc4578940bca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:34.063776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:34.066155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:34.067565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 77/152] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:49:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:34.071766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506cb86-de97-4493-bf9f-bc0e8dbedc6e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:34.134732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:34.288293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506cb86-de97-4493-bf9f-bc0e8dbedc6e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:34.359830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d285f7-9fb7-48cb-babb-3ee61c5a486f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:34.361553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d285f7-9fb7-48cb-babb-3ee61c5a486f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:34.362090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d285f7-9fb7-48cb-babb-3ee61c5a486f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:34.362579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d285f7-9fb7-48cb-babb-3ee61c5a486f None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:34.363304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167d48f-9568-4f86-b092-57a4a7e50a53 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:34.363603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167d48f-9568-4f86-b092-57a4a7e50a53 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:34.363886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167d48f-9568-4f86-b092-57a4a7e50a53 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:34.364051 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d285f7-9fb7-48cb-babb-3ee61c5a486f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:34.364510 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d285f7-9fb7-48cb-babb-3ee61c5a486f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:34.365514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 78/153] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:34.366926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d285f7-9fb7-48cb-babb-3ee61c5a486f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:34.372233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec356a9-e1d5-4bd5-bae1-a3b57cd747f1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:34.372584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec356a9-e1d5-4bd5-bae1-a3b57cd747f1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:34.372879 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec356a9-e1d5-4bd5-bae1-a3b57cd747f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:34.385575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 76/154] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:49:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:34.395442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d285f7-9fb7-48cb-babb-3ee61c5a486f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:34.408904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9ec356a9-e1d5-4bd5-bae1-a3b57cd747f1 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:34.410529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9ec356a9-e1d5-4bd5-bae1-a3b57cd747f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:34.510207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 79/155] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:34.532230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec356a9-e1d5-4bd5-bae1-a3b57cd747f1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:34.567878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:34.596987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d285f7-9fb7-48cb-babb-3ee61c5a486f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:34.671924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818961e-9b51-4716-aea9-15972643178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:34.673750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818961e-9b51-4716-aea9-15972643178b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:34.674374 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818961e-9b51-4716-aea9-15972643178b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:34.674954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818961e-9b51-4716-aea9-15972643178b None admin] PATH_INFO: `/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:34.677170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818961e-9b51-4716-aea9-15972643178b None admin] RBAC: Authorizing `identity:get_user(user_id=6c3cd73e4b2b4c3e83764570cbced9fa)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:34.679938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818961e-9b51-4716-aea9-15972643178b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:34.689661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 77/156] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:34 2022] GET /identity/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:34.692315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818961e-9b51-4716-aea9-15972643178b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:34.701153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec356a9-e1d5-4bd5-bae1-a3b57cd747f1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:34.724783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:34.746437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5372a01-8dbb-45f9-827e-f4d5f816c32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:34.747383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5372a01-8dbb-45f9-827e-f4d5f816c32b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:34.747732 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5372a01-8dbb-45f9-827e-f4d5f816c32b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:34.748075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5372a01-8dbb-45f9-827e-f4d5f816c32b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:34.763848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5372a01-8dbb-45f9-827e-f4d5f816c32b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b5372a01-8dbb-45f9-827e-f4d5f816c32b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:34.773230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:34.775581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:34.777103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 80/157] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:49:34.781997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5372a01-8dbb-45f9-827e-f4d5f816c32b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:34.930330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5372a01-8dbb-45f9-827e-f4d5f816c32b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:34.953423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818961e-9b51-4716-aea9-15972643178b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:35.002572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a68ed-6f50-4fcd-b765-8ffe92f6af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:35.004711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a68ed-6f50-4fcd-b765-8ffe92f6af26 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:35.007505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a68ed-6f50-4fcd-b765-8ffe92f6af26 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:35.009095 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a68ed-6f50-4fcd-b765-8ffe92f6af26 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:35.012254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a68ed-6f50-4fcd-b765-8ffe92f6af26 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:35.013409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a68ed-6f50-4fcd-b765-8ffe92f6af26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:35.019003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a68ed-6f50-4fcd-b765-8ffe92f6af26 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:35.048498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbd70e-639d-4b6e-b37a-ce2511ee84a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:35.052725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 81/158] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:35.054466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbd70e-639d-4b6e-b37a-ce2511ee84a6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:35.055407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbd70e-639d-4b6e-b37a-ce2511ee84a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:35.056235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbd70e-639d-4b6e-b37a-ce2511ee84a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:35.063771 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a68ed-6f50-4fcd-b765-8ffe92f6af26 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:35.126536 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:35.164314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:35.222054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dbd70e-639d-4b6e-b37a-ce2511ee84a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:35.224231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dbd70e-639d-4b6e-b37a-ce2511ee84a6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:35.303737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a68ed-6f50-4fcd-b765-8ffe92f6af26 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:35.385487 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929f0e3-5b9d-413f-a652-115e35a92d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:35.386810 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929f0e3-5b9d-413f-a652-115e35a92d47 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:35.393439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929f0e3-5b9d-413f-a652-115e35a92d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:35.394123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929f0e3-5b9d-413f-a652-115e35a92d47 None admin] PATH_INFO: `/v3/users/4d814c6361c14ec08243293cf4dec26a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:35.396077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929f0e3-5b9d-413f-a652-115e35a92d47 None admin] RBAC: Authorizing `identity:get_user(user_id=4d814c6361c14ec08243293cf4dec26a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:35.397978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929f0e3-5b9d-413f-a652-115e35a92d47 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:35.400383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 82/159] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:35 2022] GET /identity/v3/users/4d814c6361c14ec08243293cf4dec26a => generated 289 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:35.428552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929f0e3-5b9d-413f-a652-115e35a92d47 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:35.454771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 78/160] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 763 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 07:49:35.460746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbd70e-639d-4b6e-b37a-ce2511ee84a6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:35.579256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:35.646971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929f0e3-5b9d-413f-a652-115e35a92d47 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:35.682570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbd70e-639d-4b6e-b37a-ce2511ee84a6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:35.686029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:35.727598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df0a67-6661-49b6-a642-89695f7e44fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:35.729180 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df0a67-6661-49b6-a642-89695f7e44fa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:35.730849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df0a67-6661-49b6-a642-89695f7e44fa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:35.732092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df0a67-6661-49b6-a642-89695f7e44fa None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:35.734462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df0a67-6661-49b6-a642-89695f7e44fa None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:35.737753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df0a67-6661-49b6-a642-89695f7e44fa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:35.740419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 83/161] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:35 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:35.744544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff8210-4725-4c79-bbe6-fbd4e90118c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:35.745851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff8210-4725-4c79-bbe6-fbd4e90118c5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:35.746354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff8210-4725-4c79-bbe6-fbd4e90118c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:35.746879 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff8210-4725-4c79-bbe6-fbd4e90118c5 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:35.748532 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff8210-4725-4c79-bbe6-fbd4e90118c5 None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:35.749107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21290d4-8e77-4c25-a501-5c0650722cda None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:35.749667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21290d4-8e77-4c25-a501-5c0650722cda None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:35.750141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21290d4-8e77-4c25-a501-5c0650722cda None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:35.751888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 84/162] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:35.753453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d8e98-b3d0-4f1d-8dc6-3633575df5de None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:35.754348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d8e98-b3d0-4f1d-8dc6-3633575df5de None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:35.754838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d8e98-b3d0-4f1d-8dc6-3633575df5de None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:35.755098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff8210-4725-4c79-bbe6-fbd4e90118c5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:35.756743 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 85/163] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:35.761690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 79/164] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:35 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:35.762338 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d8e98-b3d0-4f1d-8dc6-3633575df5de None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:35.767661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6cc49-f20d-44be-948e-0cb6113a7042 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:35.767661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6cc49-f20d-44be-948e-0cb6113a7042 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:35.773656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6cc49-f20d-44be-948e-0cb6113a7042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:35.812733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a1e6cc49-f20d-44be-948e-0cb6113a7042 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:35.814267 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a1e6cc49-f20d-44be-948e-0cb6113a7042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:35.900999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d8e98-b3d0-4f1d-8dc6-3633575df5de None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:35.941815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23a5cf-6f54-46b8-8ca1-82dc3fa52917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:35.943627 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23a5cf-6f54-46b8-8ca1-82dc3fa52917 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:35.944111 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23a5cf-6f54-46b8-8ca1-82dc3fa52917 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:35.944764 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23a5cf-6f54-46b8-8ca1-82dc3fa52917 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca/users/4d814c6361c14ec08243293cf4dec26a/roles/48f409e647e64284817bad2b8ec30bf2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:35.947780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23a5cf-6f54-46b8-8ca1-82dc3fa52917 None admin] RBAC: Authorizing `identity:create_grant(project_id=d44ad181a25048dea48323087ad2f3ca, user_id=4d814c6361c14ec08243293cf4dec26a, role_id=48f409e647e64284817bad2b8ec30bf2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:35.949672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23a5cf-6f54-46b8-8ca1-82dc3fa52917 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:35.966630 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 86/165] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:49:35 2022] PUT /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca/users/4d814c6361c14ec08243293cf4dec26a/roles/48f409e647e64284817bad2b8ec30bf2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:49:35.968062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23a5cf-6f54-46b8-8ca1-82dc3fa52917 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:36.004521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 80/166] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:36.005906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcae286-dcd2-4810-8cfe-0d7afa1e637a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:36.006208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcae286-dcd2-4810-8cfe-0d7afa1e637a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:36.006471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcae286-dcd2-4810-8cfe-0d7afa1e637a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:36.016125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:36.046752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-adcae286-dcd2-4810-8cfe-0d7afa1e637a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:36.047995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-adcae286-dcd2-4810-8cfe-0d7afa1e637a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:36.104541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23a5cf-6f54-46b8-8ca1-82dc3fa52917 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:36.150681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48e961-6842-478f-966a-cf812b2920c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:36.152061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48e961-6842-478f-966a-cf812b2920c8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:36.152580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48e961-6842-478f-966a-cf812b2920c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:36.153134 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48e961-6842-478f-966a-cf812b2920c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:36.154364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48e961-6842-478f-966a-cf812b2920c8 None admin] RBAC: Adding query filter params (role.id=794ff5f5032d40a98e67fb0df718b75c, scope.project.id=d44ad181a25048dea48323087ad2f3ca, user.id=6c3cd73e4b2b4c3e83764570cbced9fa) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:36.155022 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48e961-6842-478f-966a-cf812b2920c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:36.156766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48e961-6842-478f-966a-cf812b2920c8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:36.164449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 87/167] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:35 2022] GET /identity/v3/role_assignments?user.id=6c3cd73e4b2b4c3e83764570cbced9fa&scope.project.id=d44ad181a25048dea48323087ad2f3ca&role.id=794ff5f5032d40a98e67fb0df718b75c => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:36.166016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48e961-6842-478f-966a-cf812b2920c8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:36.196721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:36.268410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 81/168] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:36.310928 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:36.315922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48e961-6842-478f-966a-cf812b2920c8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:36.373107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbeea3-dd9f-4324-be91-6b7f098b9c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:36.374543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbeea3-dd9f-4324-be91-6b7f098b9c65 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:36.375041 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbeea3-dd9f-4324-be91-6b7f098b9c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:36.375518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbeea3-dd9f-4324-be91-6b7f098b9c65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:36.382846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcae286-dcd2-4810-8cfe-0d7afa1e637a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:36.392653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbeea3-dd9f-4324-be91-6b7f098b9c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e9bbeea3-dd9f-4324-be91-6b7f098b9c65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:36.402251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:36.403416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:36.404238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:36.404238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:36.404238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:36.404238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:36.404238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:36.404238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:36.404238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:36.404238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:36.404238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:36.404238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:36.404238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:36.404238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:36.407174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 88/169] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:36.415597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbeea3-dd9f-4324-be91-6b7f098b9c65 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:36.644690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcae286-dcd2-4810-8cfe-0d7afa1e637a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:36.672697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbeea3-dd9f-4324-be91-6b7f098b9c65 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:36.718266 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b56c9-34ae-40af-b428-9856acdbbec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:36.719108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b56c9-34ae-40af-b428-9856acdbbec1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:36.719393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b56c9-34ae-40af-b428-9856acdbbec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:36.719669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b56c9-34ae-40af-b428-9856acdbbec1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:36.739703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b56c9-34ae-40af-b428-9856acdbbec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-034b56c9-34ae-40af-b428-9856acdbbec1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:36.749086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:36.750074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:36.750981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:36.750981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:36.750981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:36.750981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:36.750981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:36.750981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:36.750981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:36.750981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:36.750981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:36.750981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:36.750981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:36.750981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:36.751591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba3a3a-ddc3-4fbf-b7c7-4f5bd19ad184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:36.751591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba3a3a-ddc3-4fbf-b7c7-4f5bd19ad184 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:36.751591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba3a3a-ddc3-4fbf-b7c7-4f5bd19ad184 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:36.751591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba3a3a-ddc3-4fbf-b7c7-4f5bd19ad184 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:36.752432 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba3a3a-ddc3-4fbf-b7c7-4f5bd19ad184 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:36.753991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba3a3a-ddc3-4fbf-b7c7-4f5bd19ad184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:36.754459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba3a3a-ddc3-4fbf-b7c7-4f5bd19ad184 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:36.761102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 82/170] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:49:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:36.764915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 89/171] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:36.768785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b56c9-34ae-40af-b428-9856acdbbec1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:36.772725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba3a3a-ddc3-4fbf-b7c7-4f5bd19ad184 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:36.820022 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:36.851762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:36.959935 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b56c9-34ae-40af-b428-9856acdbbec1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:36.964705 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba3a3a-ddc3-4fbf-b7c7-4f5bd19ad184 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:37.029151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd670df-6d3b-40b9-8b5b-ed1fe6876c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:37.030085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd670df-6d3b-40b9-8b5b-ed1fe6876c0d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:37.030378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd670df-6d3b-40b9-8b5b-ed1fe6876c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:37.030646 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd670df-6d3b-40b9-8b5b-ed1fe6876c0d None admin] PATH_INFO: `/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:37.032410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4fa04-1aad-4ad0-a8ce-eb5531b59c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:37.033105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4fa04-1aad-4ad0-a8ce-eb5531b59c58 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:37.033387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4fa04-1aad-4ad0-a8ce-eb5531b59c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:37.033681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4fa04-1aad-4ad0-a8ce-eb5531b59c58 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:37.034453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd670df-6d3b-40b9-8b5b-ed1fe6876c0d None admin] RBAC: Authorizing `identity:get_user(user_id=6c3cd73e4b2b4c3e83764570cbced9fa)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:37.034559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4fa04-1aad-4ad0-a8ce-eb5531b59c58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:37.034761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4fa04-1aad-4ad0-a8ce-eb5531b59c58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:37.035961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd670df-6d3b-40b9-8b5b-ed1fe6876c0d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:37.036166 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4fa04-1aad-4ad0-a8ce-eb5531b59c58 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:37.038281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 90/172] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:36 2022] GET /identity/v3/users/6c3cd73e4b2b4c3e83764570cbced9fa => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:37.047137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd670df-6d3b-40b9-8b5b-ed1fe6876c0d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:37.060395 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 83/173] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:37.066346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4fa04-1aad-4ad0-a8ce-eb5531b59c58 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:37.190346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4fa04-1aad-4ad0-a8ce-eb5531b59c58 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:37.192193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:37.235419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1861cd-e4ae-4db2-8eba-60cc22f5a6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:37.236787 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1861cd-e4ae-4db2-8eba-60cc22f5a6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:37.237281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1861cd-e4ae-4db2-8eba-60cc22f5a6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:37.237790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1861cd-e4ae-4db2-8eba-60cc22f5a6e4 None admin] PATH_INFO: `/v3/users/aec7b27e85c74490a82ad099b2e0743e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:37.239338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1861cd-e4ae-4db2-8eba-60cc22f5a6e4 None admin] RBAC: Authorizing `identity:get_user(user_id=aec7b27e85c74490a82ad099b2e0743e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:37.241077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1861cd-e4ae-4db2-8eba-60cc22f5a6e4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:37.242002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:37.245000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 84/174] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:37 2022] GET /identity/v3/users/aec7b27e85c74490a82ad099b2e0743e => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:37.250788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1861cd-e4ae-4db2-8eba-60cc22f5a6e4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:37.253224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd670df-6d3b-40b9-8b5b-ed1fe6876c0d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:37.317248 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e16f82-a339-4a7e-b1bc-b1f1725af5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:37.317954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16f82-a339-4a7e-b1bc-b1f1725af5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:37.318239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16f82-a339-4a7e-b1bc-b1f1725af5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:37.318515 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16f82-a339-4a7e-b1bc-b1f1725af5a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:37.371643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1861cd-e4ae-4db2-8eba-60cc22f5a6e4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:37.412144 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ff16b-e323-409f-90e6-71911f233609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:37.413240 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ff16b-e323-409f-90e6-71911f233609 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:37.413775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ff16b-e323-409f-90e6-71911f233609 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:37.414372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ff16b-e323-409f-90e6-71911f233609 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:37.435249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e16f82-a339-4a7e-b1bc-b1f1725af5a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:37.437033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e16f82-a339-4a7e-b1bc-b1f1725af5a1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:37.463255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:37.509333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ff16b-e323-409f-90e6-71911f233609 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:37.513326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ff16b-e323-409f-90e6-71911f233609 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:37.599580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:37.628075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 91/175] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:37.636808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e16f82-a339-4a7e-b1bc-b1f1725af5a1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:37.666706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 85/176] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 07:49:37.674464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ff16b-e323-409f-90e6-71911f233609 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:37.834275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:37.895923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e16f82-a339-4a7e-b1bc-b1f1725af5a1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:37.906983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ff16b-e323-409f-90e6-71911f233609 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:37.989122 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25272f-b5f7-4285-a934-d9bd1c4819ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:37.990809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25272f-b5f7-4285-a934-d9bd1c4819ca None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:37.991307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25272f-b5f7-4285-a934-d9bd1c4819ca None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:37.991773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25272f-b5f7-4285-a934-d9bd1c4819ca None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:37.995114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25272f-b5f7-4285-a934-d9bd1c4819ca None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:37.996043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a654de5-6e3d-49cd-91e1-58c3218ec457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:37.997523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a654de5-6e3d-49cd-91e1-58c3218ec457 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:37.998034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a654de5-6e3d-49cd-91e1-58c3218ec457 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:37.998506 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a654de5-6e3d-49cd-91e1-58c3218ec457 None admin] PATH_INFO: `/v3/projects/3ee53f9ed78b490d934f4ac8566dbcec` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:38.001646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25272f-b5f7-4285-a934-d9bd1c4819ca None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:38.002658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a654de5-6e3d-49cd-91e1-58c3218ec457 None admin] RBAC: Authorizing `identity:get_project(project_id=3ee53f9ed78b490d934f4ac8566dbcec)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:38.003389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 86/177] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:37 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:38.004948 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a654de5-6e3d-49cd-91e1-58c3218ec457 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:38.007027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 92/178] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:37 2022] GET /identity/v3/projects/3ee53f9ed78b490d934f4ac8566dbcec => generated 312 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:38.014874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25272f-b5f7-4285-a934-d9bd1c4819ca None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:38.019914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a654de5-6e3d-49cd-91e1-58c3218ec457 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:38.212897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:38.304483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a654de5-6e3d-49cd-91e1-58c3218ec457 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:38.345531 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f71f1f-3a50-4820-9b59-33a37e39d312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:38.346610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f71f1f-3a50-4820-9b59-33a37e39d312 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:38.347096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f71f1f-3a50-4820-9b59-33a37e39d312 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:38.347602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f71f1f-3a50-4820-9b59-33a37e39d312 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:38.348685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f71f1f-3a50-4820-9b59-33a37e39d312 None admin] RBAC: Adding query filter params (role.id=3edce31396af478ea1ea7b76d33da530, scope.project.id=3ee53f9ed78b490d934f4ac8566dbcec, user.id=6c3cd73e4b2b4c3e83764570cbced9fa) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:38.349200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f71f1f-3a50-4820-9b59-33a37e39d312 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:38.350995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f71f1f-3a50-4820-9b59-33a37e39d312 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:38.358879 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 93/179] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:38 2022] GET /identity/v3/role_assignments?user.id=6c3cd73e4b2b4c3e83764570cbced9fa&scope.project.id=3ee53f9ed78b490d934f4ac8566dbcec&role.id=3edce31396af478ea1ea7b76d33da530 => generated 608 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:38.360718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b697f7-1608-4d18-97ff-0d7967038db1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:38.361214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b697f7-1608-4d18-97ff-0d7967038db1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:38.361712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b697f7-1608-4d18-97ff-0d7967038db1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:38.363300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 94/180] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:38.368032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25272f-b5f7-4285-a934-d9bd1c4819ca None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:38.370576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a3e79-3778-4ead-98f2-7098a4468ec1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:38.371114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a3e79-3778-4ead-98f2-7098a4468ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:38.371596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a3e79-3778-4ead-98f2-7098a4468ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:38.392084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9b3a3e79-3778-4ead-98f2-7098a4468ec1 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:38.393709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9b3a3e79-3778-4ead-98f2-7098a4468ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:38.467481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449dd910-3336-4bd1-9e8c-4ce58c43999a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:38.468184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dd910-3336-4bd1-9e8c-4ce58c43999a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:38.468844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dd910-3336-4bd1-9e8c-4ce58c43999a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:38.468844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dd910-3336-4bd1-9e8c-4ce58c43999a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:38.469650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449dd910-3336-4bd1-9e8c-4ce58c43999a None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=d44ad181a25048dea48323087ad2f3ca, user.id=aec7b27e85c74490a82ad099b2e0743e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:38.469977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449dd910-3336-4bd1-9e8c-4ce58c43999a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:38.471553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449dd910-3336-4bd1-9e8c-4ce58c43999a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:38.480209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 95/181] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:38.485229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 87/182] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:38 2022] GET /identity/v3/role_assignments?user.id=aec7b27e85c74490a82ad099b2e0743e&scope.project.id=d44ad181a25048dea48323087ad2f3ca&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 608 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:38.540632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a3e79-3778-4ead-98f2-7098a4468ec1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:38.627586 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:38.702290 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a3e79-3778-4ead-98f2-7098a4468ec1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:38.749576 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66087c2c-40a8-4e44-b8b2-b3118909de61 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:38.749921 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66087c2c-40a8-4e44-b8b2-b3118909de61 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:38.750188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66087c2c-40a8-4e44-b8b2-b3118909de61 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:38.751966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 88/183] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:38.757604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f803e1-362b-4514-a8e1-bccf6baa3427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:38.758574 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f803e1-362b-4514-a8e1-bccf6baa3427 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:38.758574 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f803e1-362b-4514-a8e1-bccf6baa3427 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:38.759126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f803e1-362b-4514-a8e1-bccf6baa3427 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:38.770410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c0915-5a73-4f3e-a798-741e66f15633 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:38.770754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c0915-5a73-4f3e-a798-741e66f15633 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:38.771041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c0915-5a73-4f3e-a798-741e66f15633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:38.778876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:38.782667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f803e1-362b-4514-a8e1-bccf6baa3427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-d0f803e1-362b-4514-a8e1-bccf6baa3427 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:38.790733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:38.792480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:38.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 96/184] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:49:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:49:38.802411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f803e1-362b-4514-a8e1-bccf6baa3427 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:38.807116 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-875c0915-5a73-4f3e-a798-741e66f15633 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:38.808858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-875c0915-5a73-4f3e-a798-741e66f15633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:38.922045 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 89/185] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:38.946793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c0915-5a73-4f3e-a798-741e66f15633 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:38.970440 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f803e1-362b-4514-a8e1-bccf6baa3427 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:38.987288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:39.015158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b28fe2-ccb9-447c-8d85-09f74084e3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:39.016038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b28fe2-ccb9-447c-8d85-09f74084e3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:39.016193 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b28fe2-ccb9-447c-8d85-09f74084e3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:39.016505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b28fe2-ccb9-447c-8d85-09f74084e3d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:39.017191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b28fe2-ccb9-447c-8d85-09f74084e3d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:39.017512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b28fe2-ccb9-447c-8d85-09f74084e3d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:39.020202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b28fe2-ccb9-447c-8d85-09f74084e3d3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:39.030467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 97/186] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:39.036481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b28fe2-ccb9-447c-8d85-09f74084e3d3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:39.073981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c0915-5a73-4f3e-a798-741e66f15633 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:39.132152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:39.137437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36506df-ceac-4bba-ba3a-c32841b9372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:39.138772 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36506df-ceac-4bba-ba3a-c32841b9372d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:39.139145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36506df-ceac-4bba-ba3a-c32841b9372d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:39.139496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36506df-ceac-4bba-ba3a-c32841b9372d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:39.156928 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36506df-ceac-4bba-ba3a-c32841b9372d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b36506df-ceac-4bba-ba3a-c32841b9372d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:39.172734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:39.173778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:39.174690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 90/187] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:49:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:39.183158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36506df-ceac-4bba-ba3a-c32841b9372d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:39.191395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b28fe2-ccb9-447c-8d85-09f74084e3d3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:39.243594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961e428-b1f7-4d58-86e0-c54e80259a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:39.244847 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961e428-b1f7-4d58-86e0-c54e80259a49 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:39.245142 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961e428-b1f7-4d58-86e0-c54e80259a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:39.245414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961e428-b1f7-4d58-86e0-c54e80259a49 None admin] PATH_INFO: `/v3/users/f20e53c4da9242bf8b0f2d0e1e119553` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:39.267085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961e428-b1f7-4d58-86e0-c54e80259a49 None admin] RBAC: Authorizing `identity:get_user(user_id=f20e53c4da9242bf8b0f2d0e1e119553)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:39.269114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961e428-b1f7-4d58-86e0-c54e80259a49 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:39.271105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 98/188] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:39 2022] GET /identity/v3/users/f20e53c4da9242bf8b0f2d0e1e119553 => generated 283 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:39.272430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8415a5-b7b3-4d2a-999c-df819b7676d2 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:39.272719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8415a5-b7b3-4d2a-999c-df819b7676d2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:39.272998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8415a5-b7b3-4d2a-999c-df819b7676d2 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:39.275105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 99/189] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:39.282283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaa6ba-3838-48f2-9130-73defcfb236a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:39.282788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaa6ba-3838-48f2-9130-73defcfb236a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:39.283272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaa6ba-3838-48f2-9130-73defcfb236a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:39.303719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-07aaa6ba-3838-48f2-9130-73defcfb236a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:39.305035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-07aaa6ba-3838-48f2-9130-73defcfb236a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:39.339312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:39.351306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36506df-ceac-4bba-ba3a-c32841b9372d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:39.409609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45c5e7-b683-43d8-980b-07e249ccb9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:39.410864 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45c5e7-b683-43d8-980b-07e249ccb9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:39.411362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45c5e7-b683-43d8-980b-07e249ccb9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:39.411846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45c5e7-b683-43d8-980b-07e249ccb9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:39.413348 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45c5e7-b683-43d8-980b-07e249ccb9c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:39.414059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 100/190] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:39.414231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45c5e7-b683-43d8-980b-07e249ccb9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:39.415081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aaa6ba-3838-48f2-9130-73defcfb236a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:39.416695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45c5e7-b683-43d8-980b-07e249ccb9c9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:39.437341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 91/191] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:49:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:39.438431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45c5e7-b683-43d8-980b-07e249ccb9c9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:39.517042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:39.536802 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aaa6ba-3838-48f2-9130-73defcfb236a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:39.576994 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45c5e7-b683-43d8-980b-07e249ccb9c9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:39.582398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fc5ce-9828-4a88-8073-8c2ebd864630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:39.583257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fc5ce-9828-4a88-8073-8c2ebd864630 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:39.583672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fc5ce-9828-4a88-8073-8c2ebd864630 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:39.584079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fc5ce-9828-4a88-8073-8c2ebd864630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:39.624384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:39.634215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05537b2b-e049-4982-b996-e98c56e245c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:39.635612 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05537b2b-e049-4982-b996-e98c56e245c5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:39.636088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05537b2b-e049-4982-b996-e98c56e245c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:39.636552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05537b2b-e049-4982-b996-e98c56e245c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:39.655335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05537b2b-e049-4982-b996-e98c56e245c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-05537b2b-e049-4982-b996-e98c56e245c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:39.667645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:39.669315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 92/192] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:39.670525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05537b2b-e049-4982-b996-e98c56e245c5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:39.698043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fc5ce-9828-4a88-8073-8c2ebd864630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:39.700797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fc5ce-9828-4a88-8073-8c2ebd864630 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:39.834951 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 101/193] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:39.836527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63c695-577c-415f-ac12-e2d5a0087d17 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:39.836812 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63c695-577c-415f-ac12-e2d5a0087d17 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:39.837090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63c695-577c-415f-ac12-e2d5a0087d17 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:39.838721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 102/194] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:39.842616 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63c695-577c-415f-ac12-e2d5a0087d17 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:39.850450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05537b2b-e049-4982-b996-e98c56e245c5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:39.871256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:39.902165 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f5390-a7c6-4f27-848f-4eaec4cb333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:39.903491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f5390-a7c6-4f27-848f-4eaec4cb333b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:39.903963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f5390-a7c6-4f27-848f-4eaec4cb333b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:39.904415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f5390-a7c6-4f27-848f-4eaec4cb333b None admin] PATH_INFO: `/v3/users/f20e53c4da9242bf8b0f2d0e1e119553` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:39.906706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f5390-a7c6-4f27-848f-4eaec4cb333b None admin] RBAC: Authorizing `identity:get_user(user_id=f20e53c4da9242bf8b0f2d0e1e119553)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:39.909496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f5390-a7c6-4f27-848f-4eaec4cb333b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:39.918034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 93/195] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:39 2022] GET /identity/v3/users/f20e53c4da9242bf8b0f2d0e1e119553 => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:39.918034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f5390-a7c6-4f27-848f-4eaec4cb333b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:39.924432 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:39.978699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63c695-577c-415f-ac12-e2d5a0087d17 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:40.026161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ddb94-0598-44d1-a095-c26f5452ff7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:40.027312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ddb94-0598-44d1-a095-c26f5452ff7a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:40.027809 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ddb94-0598-44d1-a095-c26f5452ff7a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:40.028306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ddb94-0598-44d1-a095-c26f5452ff7a None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:40.029784 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ddb94-0598-44d1-a095-c26f5452ff7a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:40.030346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ddb94-0598-44d1-a095-c26f5452ff7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:40.033054 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ddb94-0598-44d1-a095-c26f5452ff7a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:40.048476 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f5390-a7c6-4f27-848f-4eaec4cb333b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:40.053688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 103/196] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:40.053688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86295-9ba8-44a3-91e5-c2f027053739 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:40.053688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86295-9ba8-44a3-91e5-c2f027053739 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:40.053688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86295-9ba8-44a3-91e5-c2f027053739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:40.078611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2ba86295-9ba8-44a3-91e5-c2f027053739 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:40.079852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2ba86295-9ba8-44a3-91e5-c2f027053739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:40.100465 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daa1c6-edd4-46d9-a647-8987a65abd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:40.102146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daa1c6-edd4-46d9-a647-8987a65abd6d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:40.102663 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daa1c6-edd4-46d9-a647-8987a65abd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:40.103289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daa1c6-edd4-46d9-a647-8987a65abd6d None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:40.105680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73daa1c6-edd4-46d9-a647-8987a65abd6d None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:40.109113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73daa1c6-edd4-46d9-a647-8987a65abd6d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:40.112032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 94/197] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:39 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:40.113722 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daa1c6-edd4-46d9-a647-8987a65abd6d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:40.161913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 104/198] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:40.170369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba86295-9ba8-44a3-91e5-c2f027053739 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:40.195509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:40.243914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:40.264073 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daa1c6-edd4-46d9-a647-8987a65abd6d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:40.300895 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba86295-9ba8-44a3-91e5-c2f027053739 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:40.307555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c7f59-3b14-46be-b65f-3b261097f844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:40.308229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7f59-3b14-46be-b65f-3b261097f844 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:40.308511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7f59-3b14-46be-b65f-3b261097f844 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:40.308790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7f59-3b14-46be-b65f-3b261097f844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:40.348595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10f4f0-7089-4da8-b05d-b8f59c026c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:40.349491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10f4f0-7089-4da8-b05d-b8f59c026c43 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:40.349805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10f4f0-7089-4da8-b05d-b8f59c026c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:40.350080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10f4f0-7089-4da8-b05d-b8f59c026c43 None admin] PATH_INFO: `/v3/users/4d814c6361c14ec08243293cf4dec26a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:40.351333 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10f4f0-7089-4da8-b05d-b8f59c026c43 None admin] RBAC: Authorizing `identity:get_user(user_id=4d814c6361c14ec08243293cf4dec26a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:40.353233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10f4f0-7089-4da8-b05d-b8f59c026c43 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:40.355034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 105/199] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:40 2022] GET /identity/v3/users/4d814c6361c14ec08243293cf4dec26a => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:40.356005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10f4f0-7089-4da8-b05d-b8f59c026c43 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:40.397280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7f59-3b14-46be-b65f-3b261097f844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:40.399025 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7f59-3b14-46be-b65f-3b261097f844 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:40.471426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:40.486603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10f4f0-7089-4da8-b05d-b8f59c026c43 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:40.521793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:40.533115 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 95/200] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:40.534019 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c7f59-3b14-46be-b65f-3b261097f844 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:40.536033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae29c4b-6809-4004-b474-39b22ff574e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:40.536723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae29c4b-6809-4004-b474-39b22ff574e5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:40.537001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae29c4b-6809-4004-b474-39b22ff574e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:40.537267 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae29c4b-6809-4004-b474-39b22ff574e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:40.538570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae29c4b-6809-4004-b474-39b22ff574e5 None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=e443cfb7bf354891a724342b87c62cda, user.id=f20e53c4da9242bf8b0f2d0e1e119553) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:40.538570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae29c4b-6809-4004-b474-39b22ff574e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:40.540089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae29c4b-6809-4004-b474-39b22ff574e5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:40.546536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 106/201] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:40 2022] GET /identity/v3/role_assignments?user.id=f20e53c4da9242bf8b0f2d0e1e119553&scope.project.id=e443cfb7bf354891a724342b87c62cda&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:40.547372 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae29c4b-6809-4004-b474-39b22ff574e5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:40.709594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c7f59-3b14-46be-b65f-3b261097f844 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:40.720017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae29c4b-6809-4004-b474-39b22ff574e5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:40.759309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ca36f-b1c9-4bef-82c9-fa9ea4d3ad12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:40.760402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ca36f-b1c9-4bef-82c9-fa9ea4d3ad12 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:40.760813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ca36f-b1c9-4bef-82c9-fa9ea4d3ad12 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:40.761497 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ca36f-b1c9-4bef-82c9-fa9ea4d3ad12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:40.763069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ca36f-b1c9-4bef-82c9-fa9ea4d3ad12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:40.764897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ca36f-b1c9-4bef-82c9-fa9ea4d3ad12 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:40.767357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 96/202] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:40.768209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ca36f-b1c9-4bef-82c9-fa9ea4d3ad12 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:40.776342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b921c4f-a315-43e9-ae75-09c914f0c5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:40.776990 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b921c4f-a315-43e9-ae75-09c914f0c5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:40.777340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b921c4f-a315-43e9-ae75-09c914f0c5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:40.777656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b921c4f-a315-43e9-ae75-09c914f0c5a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:40.793427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:40.866581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:40.896234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b921c4f-a315-43e9-ae75-09c914f0c5a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:40.899049 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b921c4f-a315-43e9-ae75-09c914f0c5a5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:40.913907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ca36f-b1c9-4bef-82c9-fa9ea4d3ad12 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:40.955129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b0148-7e1d-4d0d-90f1-03db40e12524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:40.956334 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b0148-7e1d-4d0d-90f1-03db40e12524 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:40.956785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b0148-7e1d-4d0d-90f1-03db40e12524 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:40.957265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b0148-7e1d-4d0d-90f1-03db40e12524 None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:40.958719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b0148-7e1d-4d0d-90f1-03db40e12524 None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:40.960606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b0148-7e1d-4d0d-90f1-03db40e12524 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:40.963021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 97/203] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:40 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:40.963978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b0148-7e1d-4d0d-90f1-03db40e12524 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:41.061122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 107/204] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:41.062440 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b921c4f-a315-43e9-ae75-09c914f0c5a5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:41.083622 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:41.123993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b0148-7e1d-4d0d-90f1-03db40e12524 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:41.161660 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb518e4-6f00-49d5-932a-728e69e15e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:41.163380 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb518e4-6f00-49d5-932a-728e69e15e13 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:41.163722 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb518e4-6f00-49d5-932a-728e69e15e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:41.164036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb518e4-6f00-49d5-932a-728e69e15e13 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:41.165742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb518e4-6f00-49d5-932a-728e69e15e13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:41.168104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb518e4-6f00-49d5-932a-728e69e15e13 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:41.199435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:41.209909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 98/205] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:49:40 2022] POST /identity/v3/users => generated 297 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:49:41.209909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb518e4-6f00-49d5-932a-728e69e15e13 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:41.224219 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b921c4f-a315-43e9-ae75-09c914f0c5a5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:41.317446 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d322d-7669-474d-a41a-ea35fb4f8b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:41.318867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d322d-7669-474d-a41a-ea35fb4f8b21 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:41.319155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d322d-7669-474d-a41a-ea35fb4f8b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:41.319573 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d322d-7669-474d-a41a-ea35fb4f8b21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:41.322565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d322d-7669-474d-a41a-ea35fb4f8b21 None admin] RBAC: Adding query filter params (role.id=794ff5f5032d40a98e67fb0df718b75c, scope.project.id=e443cfb7bf354891a724342b87c62cda, user.id=f20e53c4da9242bf8b0f2d0e1e119553) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:41.323074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d322d-7669-474d-a41a-ea35fb4f8b21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:41.325970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d322d-7669-474d-a41a-ea35fb4f8b21 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:41.342240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 108/206] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:41 2022] GET /identity/v3/role_assignments?user.id=f20e53c4da9242bf8b0f2d0e1e119553&scope.project.id=e443cfb7bf354891a724342b87c62cda&role.id=794ff5f5032d40a98e67fb0df718b75c => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:41.450276 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb518e4-6f00-49d5-932a-728e69e15e13 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:41.514926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc90844-7fec-4a28-b3f7-20d1e924fb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:41.516376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc90844-7fec-4a28-b3f7-20d1e924fb6c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:41.516949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc90844-7fec-4a28-b3f7-20d1e924fb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:41.517451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc90844-7fec-4a28-b3f7-20d1e924fb6c None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:41.519753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc90844-7fec-4a28-b3f7-20d1e924fb6c None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:41.522527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc90844-7fec-4a28-b3f7-20d1e924fb6c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:41.527108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 99/207] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:41 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:41.537035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d322d-7669-474d-a41a-ea35fb4f8b21 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:41.714839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d322d-7669-474d-a41a-ea35fb4f8b21 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:41.761690 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6a208-8244-4e2b-9b33-2ebb7d825cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:41.762386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6a208-8244-4e2b-9b33-2ebb7d825cae None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:41.762669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6a208-8244-4e2b-9b33-2ebb7d825cae None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:41.762944 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6a208-8244-4e2b-9b33-2ebb7d825cae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:41.763824 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6a208-8244-4e2b-9b33-2ebb7d825cae None admin] RBAC: Adding query filter params (role.id=48f409e647e64284817bad2b8ec30bf2, scope.project.id=d44ad181a25048dea48323087ad2f3ca, user.id=4d814c6361c14ec08243293cf4dec26a) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:41.764136 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6a208-8244-4e2b-9b33-2ebb7d825cae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:41.765648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6a208-8244-4e2b-9b33-2ebb7d825cae None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:41.773126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 109/208] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:41 2022] GET /identity/v3/role_assignments?user.id=4d814c6361c14ec08243293cf4dec26a&scope.project.id=d44ad181a25048dea48323087ad2f3ca&role.id=48f409e647e64284817bad2b8ec30bf2 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:42.745285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481a22f-10ae-45b2-9c9f-05da295f2c1d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:42.746103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481a22f-10ae-45b2-9c9f-05da295f2c1d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:42.746103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481a22f-10ae-45b2-9c9f-05da295f2c1d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:42.747289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 100/209] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:42.753307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c963176-263a-458b-99ba-af94d8e9952c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:42.754596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c963176-263a-458b-99ba-af94d8e9952c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:42.755068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c963176-263a-458b-99ba-af94d8e9952c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:42.773364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0c963176-263a-458b-99ba-af94d8e9952c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:42.775168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0c963176-263a-458b-99ba-af94d8e9952c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:42.787638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:42.904476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 110/210] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:42.926172 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481a22f-10ae-45b2-9c9f-05da295f2c1d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:42.940959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:43.045712 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481a22f-10ae-45b2-9c9f-05da295f2c1d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:43.082171 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f14f7-3665-4c01-bbc2-ae43394da145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:43.083395 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f14f7-3665-4c01-bbc2-ae43394da145 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:43.083891 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f14f7-3665-4c01-bbc2-ae43394da145 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:43.084356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f14f7-3665-4c01-bbc2-ae43394da145 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:43.096574 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f14f7-3665-4c01-bbc2-ae43394da145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-989f14f7-3665-4c01-bbc2-ae43394da145 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:43.105066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:43.106063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:43.106886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:43.106886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:43.106886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:43.106886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:43.106886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:43.106886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:43.106886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:43.106886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:43.106886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:43.106886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:43.106886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:43.106886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:43.118340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 101/211] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:49:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:43.120247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c963176-263a-458b-99ba-af94d8e9952c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:43.239808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c963176-263a-458b-99ba-af94d8e9952c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:43.280297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187fce5-7a5f-4d2c-958e-3da7b7bfea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:43.281373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187fce5-7a5f-4d2c-958e-3da7b7bfea5a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:43.281902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187fce5-7a5f-4d2c-958e-3da7b7bfea5a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:43.282387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187fce5-7a5f-4d2c-958e-3da7b7bfea5a None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:43.283346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187fce5-7a5f-4d2c-958e-3da7b7bfea5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:43.283864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187fce5-7a5f-4d2c-958e-3da7b7bfea5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:43.285486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187fce5-7a5f-4d2c-958e-3da7b7bfea5a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:43.295851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 111/212] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:43.301953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f14f7-3665-4c01-bbc2-ae43394da145 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:43.380697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:43.415610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f14f7-3665-4c01-bbc2-ae43394da145 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:43.452665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32996f96-4b46-450e-8146-9913e5113a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:43.453836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32996f96-4b46-450e-8146-9913e5113a8e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:43.454123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32996f96-4b46-450e-8146-9913e5113a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:43.454397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32996f96-4b46-450e-8146-9913e5113a8e None admin] PATH_INFO: `/v3/users/f20e53c4da9242bf8b0f2d0e1e119553` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:43.455654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32996f96-4b46-450e-8146-9913e5113a8e None admin] RBAC: Authorizing `identity:get_user(user_id=f20e53c4da9242bf8b0f2d0e1e119553)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:43.457149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32996f96-4b46-450e-8146-9913e5113a8e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:43.458889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 102/213] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:43 2022] GET /identity/v3/users/f20e53c4da9242bf8b0f2d0e1e119553 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:43.465123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187fce5-7a5f-4d2c-958e-3da7b7bfea5a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:43.474803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:43.579800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187fce5-7a5f-4d2c-958e-3da7b7bfea5a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:43.607137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfde7b-4169-4d08-a58f-1d3abdd3c64c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:43.608048 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfde7b-4169-4d08-a58f-1d3abdd3c64c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:43.608520 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfde7b-4169-4d08-a58f-1d3abdd3c64c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:43.610939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 103/214] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:49:43.617289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701a281-f1d2-472a-a272-48d3a062a2be None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:43.617938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701a281-f1d2-472a-a272-48d3a062a2be None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:43.618114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41efaea-1042-40e1-93f1-ca4e9d74c55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:43.618605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701a281-f1d2-472a-a272-48d3a062a2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:43.618974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41efaea-1042-40e1-93f1-ca4e9d74c55a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:43.619253 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41efaea-1042-40e1-93f1-ca4e9d74c55a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:43.619524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41efaea-1042-40e1-93f1-ca4e9d74c55a None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:43.620992 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41efaea-1042-40e1-93f1-ca4e9d74c55a None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:43.622534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41efaea-1042-40e1-93f1-ca4e9d74c55a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:43.624395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 112/215] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:43 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:43.630734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41efaea-1042-40e1-93f1-ca4e9d74c55a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:43.640175 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f701a281-f1d2-472a-a272-48d3a062a2be None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:43.642614 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f701a281-f1d2-472a-a272-48d3a062a2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:43.716368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:43.740848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 104/216] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:43.749919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99faa27-8213-40c4-b137-41b7018eb76f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:43.749919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99faa27-8213-40c4-b137-41b7018eb76f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:43.749919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99faa27-8213-40c4-b137-41b7018eb76f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:43.749919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 105/217] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:43.750681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41efaea-1042-40e1-93f1-ca4e9d74c55a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:43.751812 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a348de-6330-4530-90f4-b4350a20e87a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:43.752109 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a348de-6330-4530-90f4-b4350a20e87a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:43.752387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a348de-6330-4530-90f4-b4350a20e87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:43.770765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-46a348de-6330-4530-90f4-b4350a20e87a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:43.771974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-46a348de-6330-4530-90f4-b4350a20e87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:43.793203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223007b2-c928-46c5-9169-7783cb3e6a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:43.794885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223007b2-c928-46c5-9169-7783cb3e6a53 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:43.795319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223007b2-c928-46c5-9169-7783cb3e6a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:43.795739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223007b2-c928-46c5-9169-7783cb3e6a53 None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda/users/f20e53c4da9242bf8b0f2d0e1e119553/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:43.798117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223007b2-c928-46c5-9169-7783cb3e6a53 None admin] RBAC: Authorizing `identity:create_grant(project_id=e443cfb7bf354891a724342b87c62cda, user_id=f20e53c4da9242bf8b0f2d0e1e119553, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:43.799806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223007b2-c928-46c5-9169-7783cb3e6a53 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:43.815757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 113/218] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:49:43 2022] PUT /identity/v3/projects/e443cfb7bf354891a724342b87c62cda/users/f20e53c4da9242bf8b0f2d0e1e119553/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:49:43.816928 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223007b2-c928-46c5-9169-7783cb3e6a53 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:43.856511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:43.863075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 106/219] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:43.888773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a348de-6330-4530-90f4-b4350a20e87a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:43.966732 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223007b2-c928-46c5-9169-7783cb3e6a53 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:44.009132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a348de-6330-4530-90f4-b4350a20e87a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:44.011333 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ee9d4-dfbe-4fb6-be19-66b775528de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:44.012205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ee9d4-dfbe-4fb6-be19-66b775528de0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:44.012489 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ee9d4-dfbe-4fb6-be19-66b775528de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:44.012763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ee9d4-dfbe-4fb6-be19-66b775528de0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:44.015233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:44.026903 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ee9d4-dfbe-4fb6-be19-66b775528de0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1d4ee9d4-dfbe-4fb6-be19-66b775528de0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:44.035565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:44.036562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:44.037850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 114/220] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:49:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:44.043453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ee9d4-dfbe-4fb6-be19-66b775528de0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:44.058841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5cf2b-b5d2-4937-9d17-9908f35124bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:44.060072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5cf2b-b5d2-4937-9d17-9908f35124bb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:44.060680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5cf2b-b5d2-4937-9d17-9908f35124bb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:44.061166 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5cf2b-b5d2-4937-9d17-9908f35124bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:44.074019 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5cf2b-b5d2-4937-9d17-9908f35124bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-58a5cf2b-b5d2-4937-9d17-9908f35124bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:44.083056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:44.084021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:44.084854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:44.084854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:44.084854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:44.084854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:44.084854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:44.084854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:44.084854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:44.084854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:44.084854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:44.084854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:44.084854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:44.084854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:44.092375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 107/221] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:49:44.094129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5cf2b-b5d2-4937-9d17-9908f35124bb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:44.169835 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:44.196532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ee9d4-dfbe-4fb6-be19-66b775528de0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:44.238803 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5cf2b-b5d2-4937-9d17-9908f35124bb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:44.240570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c5492-87c0-4db4-8df5-e956eb7fe859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:44.241285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c5492-87c0-4db4-8df5-e956eb7fe859 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:44.241573 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c5492-87c0-4db4-8df5-e956eb7fe859 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:44.242007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c5492-87c0-4db4-8df5-e956eb7fe859 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:44.242746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c5492-87c0-4db4-8df5-e956eb7fe859 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:44.243055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c5492-87c0-4db4-8df5-e956eb7fe859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:44.244864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c5492-87c0-4db4-8df5-e956eb7fe859 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:44.256057 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 115/222] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:49:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:44.266802 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c5492-87c0-4db4-8df5-e956eb7fe859 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:44.293300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bc455-0faf-4c46-bc62-a557977fe04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:44.295119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bc455-0faf-4c46-bc62-a557977fe04e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:44.295390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bc455-0faf-4c46-bc62-a557977fe04e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:44.295700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bc455-0faf-4c46-bc62-a557977fe04e None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:44.296533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bc455-0faf-4c46-bc62-a557977fe04e None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:44.296854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bc455-0faf-4c46-bc62-a557977fe04e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:44.298521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bc455-0faf-4c46-bc62-a557977fe04e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:44.313759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 108/223] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:44.319764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bc455-0faf-4c46-bc62-a557977fe04e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:44.392296 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:44.446126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c5492-87c0-4db4-8df5-e956eb7fe859 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:44.485673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97513edf-326f-4b70-ae28-0c43924eb9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:44.486626 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97513edf-326f-4b70-ae28-0c43924eb9ce None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:44.486908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97513edf-326f-4b70-ae28-0c43924eb9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:44.487183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97513edf-326f-4b70-ae28-0c43924eb9ce None admin] PATH_INFO: `/v3/users/f20e53c4da9242bf8b0f2d0e1e119553` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:44.488474 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97513edf-326f-4b70-ae28-0c43924eb9ce None admin] RBAC: Authorizing `identity:get_user(user_id=f20e53c4da9242bf8b0f2d0e1e119553)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:44.490122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97513edf-326f-4b70-ae28-0c43924eb9ce None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:44.491981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 116/224] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:44 2022] GET /identity/v3/users/f20e53c4da9242bf8b0f2d0e1e119553 => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:44.498072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97513edf-326f-4b70-ae28-0c43924eb9ce None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:44.520540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bc455-0faf-4c46-bc62-a557977fe04e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:44.559195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:44.585437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3c167-956f-48a4-9858-bb5d1460396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:44.587469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3c167-956f-48a4-9858-bb5d1460396e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:44.587963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3c167-956f-48a4-9858-bb5d1460396e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:44.588441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3c167-956f-48a4-9858-bb5d1460396e None admin] PATH_INFO: `/v3/users/0784168464c343a69d8866276838aefd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:44.615330 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3c167-956f-48a4-9858-bb5d1460396e None admin] RBAC: Authorizing `identity:get_user(user_id=0784168464c343a69d8866276838aefd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:44.618497 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3c167-956f-48a4-9858-bb5d1460396e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:44.618769 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97513edf-326f-4b70-ae28-0c43924eb9ce None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:44.622271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 109/225] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:44 2022] GET /identity/v3/users/0784168464c343a69d8866276838aefd => generated 297 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:44.627859 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3c167-956f-48a4-9858-bb5d1460396e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:44.647917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:44.663589 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e474b-da2a-4771-a07b-258dc1fbe0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:44.664811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e474b-da2a-4771-a07b-258dc1fbe0cf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:44.665096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e474b-da2a-4771-a07b-258dc1fbe0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:44.665483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e474b-da2a-4771-a07b-258dc1fbe0cf None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:44.667185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e474b-da2a-4771-a07b-258dc1fbe0cf None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:44.668747 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e474b-da2a-4771-a07b-258dc1fbe0cf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:44.670965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 117/226] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:44 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:44.677155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e474b-da2a-4771-a07b-258dc1fbe0cf None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:44.785926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3c167-956f-48a4-9858-bb5d1460396e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:44.808079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e474b-da2a-4771-a07b-258dc1fbe0cf None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:44.834317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778cab9-380d-435e-8246-435033f0356e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:44.835326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778cab9-380d-435e-8246-435033f0356e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:44.835827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778cab9-380d-435e-8246-435033f0356e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:44.836381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778cab9-380d-435e-8246-435033f0356e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:44.851795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cd1d2-02ce-4e24-ac61-b8dc5dc157a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:44.853302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cd1d2-02ce-4e24-ac61-b8dc5dc157a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:44.853838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cd1d2-02ce-4e24-ac61-b8dc5dc157a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:44.854331 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cd1d2-02ce-4e24-ac61-b8dc5dc157a4 None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda/users/f20e53c4da9242bf8b0f2d0e1e119553/roles/794ff5f5032d40a98e67fb0df718b75c` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:44.856981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5cd1d2-02ce-4e24-ac61-b8dc5dc157a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e443cfb7bf354891a724342b87c62cda, user_id=f20e53c4da9242bf8b0f2d0e1e119553, role_id=794ff5f5032d40a98e67fb0df718b75c)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:44.858805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5cd1d2-02ce-4e24-ac61-b8dc5dc157a4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:44.875449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 118/227] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:49:44 2022] PUT /identity/v3/projects/e443cfb7bf354891a724342b87c62cda/users/f20e53c4da9242bf8b0f2d0e1e119553/roles/794ff5f5032d40a98e67fb0df718b75c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 07:49:44.898668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:44.926829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778cab9-380d-435e-8246-435033f0356e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:44.928376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778cab9-380d-435e-8246-435033f0356e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:45.056344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 110/228] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:45.057871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cd1d2-02ce-4e24-ac61-b8dc5dc157a4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:45.136620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:45.214901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481c9e-1b47-4003-93d3-19810d6dc34a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:45.215303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481c9e-1b47-4003-93d3-19810d6dc34a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:45.215536 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481c9e-1b47-4003-93d3-19810d6dc34a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:45.217159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 111/229] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:49:45.225823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e29b0-fc72-47b1-8e40-da852cad5dee None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:45.226198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e29b0-fc72-47b1-8e40-da852cad5dee None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:45.226375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e29b0-fc72-47b1-8e40-da852cad5dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:45.231128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cd1d2-02ce-4e24-ac61-b8dc5dc157a4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:45.244157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ac2e29b0-fc72-47b1-8e40-da852cad5dee None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:45.245404 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ac2e29b0-fc72-47b1-8e40-da852cad5dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:45.300450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710a18d-3553-4b54-a4f0-7e5770c2c8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:45.302599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710a18d-3553-4b54-a4f0-7e5770c2c8ee None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:45.303424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710a18d-3553-4b54-a4f0-7e5770c2c8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:45.304453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710a18d-3553-4b54-a4f0-7e5770c2c8ee None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:45.306826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710a18d-3553-4b54-a4f0-7e5770c2c8ee None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:45.311273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710a18d-3553-4b54-a4f0-7e5770c2c8ee None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:45.316550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 119/230] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:45 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:45.324569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710a18d-3553-4b54-a4f0-7e5770c2c8ee None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:45.331168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 112/231] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:45.356114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e29b0-fc72-47b1-8e40-da852cad5dee None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:45.369119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:45.476554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e29b0-fc72-47b1-8e40-da852cad5dee None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:45.484382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710a18d-3553-4b54-a4f0-7e5770c2c8ee None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:45.506314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:45.515354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b7823-e5aa-455f-a61a-202936e7cec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:45.516327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b7823-e5aa-455f-a61a-202936e7cec7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:45.516595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b7823-e5aa-455f-a61a-202936e7cec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:45.516869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b7823-e5aa-455f-a61a-202936e7cec7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:45.518145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b7823-e5aa-455f-a61a-202936e7cec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:45.519613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b7823-e5aa-455f-a61a-202936e7cec7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:45.521489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 113/232] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:45.525060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d87d8-604b-4037-ab8a-1d972c3cbde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:45.526377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d87d8-604b-4037-ab8a-1d972c3cbde5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:45.526810 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d87d8-604b-4037-ab8a-1d972c3cbde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:45.527113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b7823-e5aa-455f-a61a-202936e7cec7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:45.527388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d87d8-604b-4037-ab8a-1d972c3cbde5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:45.528496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d87d8-604b-4037-ab8a-1d972c3cbde5 None admin] RBAC: Adding query filter params (role.id=3edce31396af478ea1ea7b76d33da530, scope.project.id=e443cfb7bf354891a724342b87c62cda, user.id=0784168464c343a69d8866276838aefd) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:45.528968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d87d8-604b-4037-ab8a-1d972c3cbde5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:45.530699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d87d8-604b-4037-ab8a-1d972c3cbde5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:45.544446 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 120/233] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:45 2022] GET /identity/v3/role_assignments?user.id=0784168464c343a69d8866276838aefd&scope.project.id=e443cfb7bf354891a724342b87c62cda&role.id=3edce31396af478ea1ea7b76d33da530 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:45.623844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:45.654028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b7823-e5aa-455f-a61a-202936e7cec7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:45.703937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27837f-4761-48d7-98d9-4f3e24dd3b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:45.704631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27837f-4761-48d7-98d9-4f3e24dd3b46 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:45.704906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27837f-4761-48d7-98d9-4f3e24dd3b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:45.705182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27837f-4761-48d7-98d9-4f3e24dd3b46 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:45.705949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27837f-4761-48d7-98d9-4f3e24dd3b46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:45.707456 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27837f-4761-48d7-98d9-4f3e24dd3b46 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:45.733503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 114/234] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:49:45 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 07:49:46.838981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760d4a-3234-453b-b1e4-e2e61d2eb011 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:46.839531 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760d4a-3234-453b-b1e4-e2e61d2eb011 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:46.839823 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760d4a-3234-453b-b1e4-e2e61d2eb011 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:46.842214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 121/235] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:46.858320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e5c30-f043-4089-ab9c-a31f908ddf8e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:46.858867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e5c30-f043-4089-ab9c-a31f908ddf8e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:46.859167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e5c30-f043-4089-ab9c-a31f908ddf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:46.899723 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0f3e5c30-f043-4089-ab9c-a31f908ddf8e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:46.901702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0f3e5c30-f043-4089-ab9c-a31f908ddf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:47.012216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 115/236] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:47.080612 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b760d4a-3234-453b-b1e4-e2e61d2eb011 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:47.265964 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b760d4a-3234-453b-b1e4-e2e61d2eb011 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:47.303658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a81751-64d0-4ef0-9dcc-c40e99c709d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:47.304490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a81751-64d0-4ef0-9dcc-c40e99c709d3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:47.305025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a81751-64d0-4ef0-9dcc-c40e99c709d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:47.305294 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a81751-64d0-4ef0-9dcc-c40e99c709d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:47.317319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a81751-64d0-4ef0-9dcc-c40e99c709d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-05a81751-64d0-4ef0-9dcc-c40e99c709d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:47.325632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:47.326594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:47.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:47.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:47.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:47.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:47.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:47.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:47.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:47.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:47.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:47.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:47.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:49:47.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:47.328045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 122/237] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:49:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:47.336060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e5c30-f043-4089-ab9c-a31f908ddf8e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:47.443080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:47.495565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e5c30-f043-4089-ab9c-a31f908ddf8e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:47.543858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ca0db-017e-4c5f-a6f3-0fa9c95d45e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:47.544570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ca0db-017e-4c5f-a6f3-0fa9c95d45e9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:47.544856 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ca0db-017e-4c5f-a6f3-0fa9c95d45e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:47.545133 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ca0db-017e-4c5f-a6f3-0fa9c95d45e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:47.545952 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ca0db-017e-4c5f-a6f3-0fa9c95d45e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:47.546264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ca0db-017e-4c5f-a6f3-0fa9c95d45e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:47.547694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ca0db-017e-4c5f-a6f3-0fa9c95d45e9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:47.562938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 116/238] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:47.570698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a81751-64d0-4ef0-9dcc-c40e99c709d3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:47.638750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:47.683655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b8374-a4b3-40e4-b98b-2eec0aab6855 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:47.684316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b8374-a4b3-40e4-b98b-2eec0aab6855 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:47.684800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b8374-a4b3-40e4-b98b-2eec0aab6855 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:47.686523 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 117/239] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:49:47.692355 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79466-496e-4d38-b976-9bd0d11d597a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:47.692971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79466-496e-4d38-b976-9bd0d11d597a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:47.693515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79466-496e-4d38-b976-9bd0d11d597a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:47.721924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0ad79466-496e-4d38-b976-9bd0d11d597a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:47.723518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0ad79466-496e-4d38-b976-9bd0d11d597a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:47.732226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a81751-64d0-4ef0-9dcc-c40e99c709d3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:47.848112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41248380-01f8-4fed-b0b4-79e110d8e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:47.849161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41248380-01f8-4fed-b0b4-79e110d8e7ae None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:47.849453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41248380-01f8-4fed-b0b4-79e110d8e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:47.849774 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41248380-01f8-4fed-b0b4-79e110d8e7ae None admin] PATH_INFO: `/v3/users/f20e53c4da9242bf8b0f2d0e1e119553` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:47.851079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41248380-01f8-4fed-b0b4-79e110d8e7ae None admin] RBAC: Authorizing `identity:get_user(user_id=f20e53c4da9242bf8b0f2d0e1e119553)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:47.852739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41248380-01f8-4fed-b0b4-79e110d8e7ae None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:47.858841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 123/240] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:47 2022] GET /identity/v3/users/f20e53c4da9242bf8b0f2d0e1e119553 => generated 283 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:47.865509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41248380-01f8-4fed-b0b4-79e110d8e7ae None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:47.916462 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 118/241] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:47.991937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad79466-496e-4d38-b976-9bd0d11d597a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:48.026688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41248380-01f8-4fed-b0b4-79e110d8e7ae None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:48.042253 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:48.072257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b9711-bcb7-4966-8ff9-fc35ad65d8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:48.073902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b9711-bcb7-4966-8ff9-fc35ad65d8bd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:48.074658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b9711-bcb7-4966-8ff9-fc35ad65d8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:48.075348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b9711-bcb7-4966-8ff9-fc35ad65d8bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:48.187577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:48.208811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b9711-bcb7-4966-8ff9-fc35ad65d8bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:48.212113 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b9711-bcb7-4966-8ff9-fc35ad65d8bd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:48.348805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad79466-496e-4d38-b976-9bd0d11d597a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:48.410770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 124/242] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:48.423420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b9711-bcb7-4966-8ff9-fc35ad65d8bd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:48.451705 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:48.454251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0f5a9-d4c3-4000-8ba2-801d56872d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:48.455874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0f5a9-d4c3-4000-8ba2-801d56872d4d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:48.456371 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0f5a9-d4c3-4000-8ba2-801d56872d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:48.456868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0f5a9-d4c3-4000-8ba2-801d56872d4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:48.479579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0f5a9-d4c3-4000-8ba2-801d56872d4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a3d0f5a9-d4c3-4000-8ba2-801d56872d4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:48.497112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:48.499001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:48.500336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 119/243] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:49:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 509 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:48.502090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e92e66-d248-4875-89a4-64b910591427 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:48.503009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e92e66-d248-4875-89a4-64b910591427 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:48.504070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e92e66-d248-4875-89a4-64b910591427 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:48.506952 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 120/244] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:48.517006 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ecdbd-9d63-48bf-9754-7710ad882e7a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:48.517769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ecdbd-9d63-48bf-9754-7710ad882e7a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:48.518479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ecdbd-9d63-48bf-9754-7710ad882e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:48.551273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-024ecdbd-9d63-48bf-9754-7710ad882e7a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:48.553668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-024ecdbd-9d63-48bf-9754-7710ad882e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:48.646626 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b9711-bcb7-4966-8ff9-fc35ad65d8bd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:48.700564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 121/245] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:48.705714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ecdbd-9d63-48bf-9754-7710ad882e7a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:48.741276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071864e-440d-4f4e-a917-4f8b0fa9bfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:48.744244 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071864e-440d-4f4e-a917-4f8b0fa9bfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:48.745084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071864e-440d-4f4e-a917-4f8b0fa9bfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:48.745808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071864e-440d-4f4e-a917-4f8b0fa9bfa0 None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:48.755633 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071864e-440d-4f4e-a917-4f8b0fa9bfa0 None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:48.758608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071864e-440d-4f4e-a917-4f8b0fa9bfa0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:48.762583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 125/246] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:48 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:48.772178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f8ce3-6314-4542-b59c-47fd32aea36e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:48.772178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f8ce3-6314-4542-b59c-47fd32aea36e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:48.772178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f8ce3-6314-4542-b59c-47fd32aea36e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:48.772178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 126/247] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:48.772178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f8ce3-6314-4542-b59c-47fd32aea36e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:48.784102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:48.853570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ecdbd-9d63-48bf-9754-7710ad882e7a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:48.879635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:48.895794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115407f-92ba-4af7-9673-f54849d643bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:48.896401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115407f-92ba-4af7-9673-f54849d643bf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:48.896751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115407f-92ba-4af7-9673-f54849d643bf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:48.897007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115407f-92ba-4af7-9673-f54849d643bf None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:48.897750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115407f-92ba-4af7-9673-f54849d643bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:48.898069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115407f-92ba-4af7-9673-f54849d643bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:48.899895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115407f-92ba-4af7-9673-f54849d643bf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:48.909050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 122/248] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:49:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:48.909396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f8ce3-6314-4542-b59c-47fd32aea36e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:48.910679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115407f-92ba-4af7-9673-f54849d643bf None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:48.949133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f1df8-e3d5-4ed9-9e58-42bca9fa11ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:48.949992 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f1df8-e3d5-4ed9-9e58-42bca9fa11ad None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:48.950275 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f1df8-e3d5-4ed9-9e58-42bca9fa11ad None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:48.950547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f1df8-e3d5-4ed9-9e58-42bca9fa11ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:48.967900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f1df8-e3d5-4ed9-9e58-42bca9fa11ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8a6f1df8-e3d5-4ed9-9e58-42bca9fa11ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:48.979125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:48.980303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:48.981332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 127/249] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:48.983408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f399800-a24d-49b4-a752-f472dd30ed3b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:48.983864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f399800-a24d-49b4-a752-f472dd30ed3b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:48.984145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f399800-a24d-49b4-a752-f472dd30ed3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:49.004432 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3f399800-a24d-49b4-a752-f472dd30ed3b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:49.005740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3f399800-a24d-49b4-a752-f472dd30ed3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:49.029253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115407f-92ba-4af7-9673-f54849d643bf None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:49.039185 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:49.083089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 128/250] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:49.084457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a13be-77f4-413d-87ac-69e2a36a0ab3 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:49.084750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a13be-77f4-413d-87ac-69e2a36a0ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:49.085028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a13be-77f4-413d-87ac-69e2a36a0ab3 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:49.086058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdce057-469d-46d7-a559-fea676c1b00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:49.089671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 129/251] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:49.089801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce057-469d-46d7-a559-fea676c1b00b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:49.089801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce057-469d-46d7-a559-fea676c1b00b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:49.089958 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a13be-77f4-413d-87ac-69e2a36a0ab3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:49.090041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce057-469d-46d7-a559-fea676c1b00b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:49.090041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdce057-469d-46d7-a559-fea676c1b00b None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=e443cfb7bf354891a724342b87c62cda, user.id=f20e53c4da9242bf8b0f2d0e1e119553) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:49.090041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdce057-469d-46d7-a559-fea676c1b00b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:49.090041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdce057-469d-46d7-a559-fea676c1b00b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:49.100263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 123/252] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:48 2022] GET /identity/v3/role_assignments?user.id=f20e53c4da9242bf8b0f2d0e1e119553&scope.project.id=e443cfb7bf354891a724342b87c62cda&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:49.101938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdce057-469d-46d7-a559-fea676c1b00b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:49.186627 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:49.232509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a13be-77f4-413d-87ac-69e2a36a0ab3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:49.277076 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdce057-469d-46d7-a559-fea676c1b00b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:49.294738 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf03f22-13ef-4de9-99c2-088605960f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:49.295803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf03f22-13ef-4de9-99c2-088605960f9c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:49.296225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf03f22-13ef-4de9-99c2-088605960f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:49.296516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf03f22-13ef-4de9-99c2-088605960f9c None admin] PATH_INFO: `/v3/users/f20e53c4da9242bf8b0f2d0e1e119553` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:49.298844 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf03f22-13ef-4de9-99c2-088605960f9c None admin] RBAC: Authorizing `identity:get_user(user_id=f20e53c4da9242bf8b0f2d0e1e119553)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:49.300351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf03f22-13ef-4de9-99c2-088605960f9c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:49.302210 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 130/253] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:49 2022] GET /identity/v3/users/f20e53c4da9242bf8b0f2d0e1e119553 => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:49.303482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab42a8-ea86-440e-ab26-70f90d6262e2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:49.303767 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab42a8-ea86-440e-ab26-70f90d6262e2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:49.304020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab42a8-ea86-440e-ab26-70f90d6262e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:49.322355 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-08ab42a8-ea86-440e-ab26-70f90d6262e2 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:49.323082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-08ab42a8-ea86-440e-ab26-70f90d6262e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:49.325685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965347f-62f3-4065-9b27-35e639823c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:49.326391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965347f-62f3-4065-9b27-35e639823c32 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:49.326677 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965347f-62f3-4065-9b27-35e639823c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:49.326955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965347f-62f3-4065-9b27-35e639823c32 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:49.327739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965347f-62f3-4065-9b27-35e639823c32 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:49.328051 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965347f-62f3-4065-9b27-35e639823c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:49.329476 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965347f-62f3-4065-9b27-35e639823c32 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:49.340360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 124/254] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:49.341234 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965347f-62f3-4065-9b27-35e639823c32 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:49.384652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:49.415429 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 131/255] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:49.418976 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab42a8-ea86-440e-ab26-70f90d6262e2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:49.475315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965347f-62f3-4065-9b27-35e639823c32 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:49.496143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:49.512628 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41179319-2605-4dca-a864-aab9f773f4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:49.513474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41179319-2605-4dca-a864-aab9f773f4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:49.513793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41179319-2605-4dca-a864-aab9f773f4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:49.514078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41179319-2605-4dca-a864-aab9f773f4b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:49.515408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41179319-2605-4dca-a864-aab9f773f4b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:49.516955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41179319-2605-4dca-a864-aab9f773f4b9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:49.518825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 125/256] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:49.519676 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41179319-2605-4dca-a864-aab9f773f4b9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:49.548897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab42a8-ea86-440e-ab26-70f90d6262e2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:49.591034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149ffe5-be44-4612-a2ed-b0e3d09a835d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:49.591722 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149ffe5-be44-4612-a2ed-b0e3d09a835d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:49.592014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149ffe5-be44-4612-a2ed-b0e3d09a835d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:49.592291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149ffe5-be44-4612-a2ed-b0e3d09a835d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:49.654610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:49.658033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41179319-2605-4dca-a864-aab9f773f4b9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:49.679231 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149ffe5-be44-4612-a2ed-b0e3d09a835d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:49.681140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149ffe5-be44-4612-a2ed-b0e3d09a835d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:49.711864 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd283d1-1ec3-4b8f-b4f2-76fad3029f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:49.712762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd283d1-1ec3-4b8f-b4f2-76fad3029f97 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:49.713046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd283d1-1ec3-4b8f-b4f2-76fad3029f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:49.713321 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd283d1-1ec3-4b8f-b4f2-76fad3029f97 None admin] PATH_INFO: `/v3/users/0784168464c343a69d8866276838aefd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:49.714657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd283d1-1ec3-4b8f-b4f2-76fad3029f97 None admin] RBAC: Authorizing `identity:get_user(user_id=0784168464c343a69d8866276838aefd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:49.716154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd283d1-1ec3-4b8f-b4f2-76fad3029f97 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:49.717942 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 126/257] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:49 2022] GET /identity/v3/users/0784168464c343a69d8866276838aefd => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:49.718753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd283d1-1ec3-4b8f-b4f2-76fad3029f97 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:49.766813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:49.819821 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 132/258] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:49.821728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149ffe5-be44-4612-a2ed-b0e3d09a835d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:49.833956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd283d1-1ec3-4b8f-b4f2-76fad3029f97 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:49.878487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712bc9a-28a7-48fe-819d-6fc89060162b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:49.879970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712bc9a-28a7-48fe-819d-6fc89060162b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:49.880413 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712bc9a-28a7-48fe-819d-6fc89060162b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:49.880798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712bc9a-28a7-48fe-819d-6fc89060162b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:49.893132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712bc9a-28a7-48fe-819d-6fc89060162b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1712bc9a-28a7-48fe-819d-6fc89060162b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:49.901582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:49.902901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 127/259] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:49.904512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712bc9a-28a7-48fe-819d-6fc89060162b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:49.939241 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:49.954639 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149ffe5-be44-4612-a2ed-b0e3d09a835d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:50.002123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea31ee6-c6d7-4c74-91ab-6bff7fbc5ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:50.003147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea31ee6-c6d7-4c74-91ab-6bff7fbc5ebc None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:50.003457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea31ee6-c6d7-4c74-91ab-6bff7fbc5ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:50.004292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea31ee6-c6d7-4c74-91ab-6bff7fbc5ebc None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:50.005718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea31ee6-c6d7-4c74-91ab-6bff7fbc5ebc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:50.008692 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea31ee6-c6d7-4c74-91ab-6bff7fbc5ebc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:50.022170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712bc9a-28a7-48fe-819d-6fc89060162b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:50.034995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 133/260] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:49:49 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:49:50.042792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea31ee6-c6d7-4c74-91ab-6bff7fbc5ebc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:50.066754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476cde5-3489-43d4-88c9-3f2209e71f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:50.067858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476cde5-3489-43d4-88c9-3f2209e71f84 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:50.068218 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476cde5-3489-43d4-88c9-3f2209e71f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:50.068663 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476cde5-3489-43d4-88c9-3f2209e71f84 None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:50.071257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476cde5-3489-43d4-88c9-3f2209e71f84 None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:50.074315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476cde5-3489-43d4-88c9-3f2209e71f84 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:50.078297 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 128/261] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:49 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:50.079749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476cde5-3489-43d4-88c9-3f2209e71f84 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:50.080641 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:50.201518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea31ee6-c6d7-4c74-91ab-6bff7fbc5ebc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:50.263949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccefba3-2ec9-49fb-a2ca-8f0ed007186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:50.265491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccefba3-2ec9-49fb-a2ca-8f0ed007186e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:50.266042 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccefba3-2ec9-49fb-a2ca-8f0ed007186e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:50.266546 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccefba3-2ec9-49fb-a2ca-8f0ed007186e None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:50.267904 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:50.268807 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccefba3-2ec9-49fb-a2ca-8f0ed007186e None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:50.271937 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccefba3-2ec9-49fb-a2ca-8f0ed007186e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:50.276963 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 134/262] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:50 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:50.281789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccefba3-2ec9-49fb-a2ca-8f0ed007186e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:50.281965 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476cde5-3489-43d4-88c9-3f2209e71f84 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:50.339599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f30073-bf29-48bf-b09a-06a95b7b4c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:50.340702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f30073-bf29-48bf-b09a-06a95b7b4c24 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:50.341203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f30073-bf29-48bf-b09a-06a95b7b4c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:50.341648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f30073-bf29-48bf-b09a-06a95b7b4c24 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:50.342966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f30073-bf29-48bf-b09a-06a95b7b4c24 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:50.343489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f30073-bf29-48bf-b09a-06a95b7b4c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:50.346054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f30073-bf29-48bf-b09a-06a95b7b4c24 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:50.363426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 129/263] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:50.364665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f30073-bf29-48bf-b09a-06a95b7b4c24 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:50.433937 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:50.454716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccefba3-2ec9-49fb-a2ca-8f0ed007186e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:50.496477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566b1b0-d739-4163-9c6b-5118f24bfd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:50.497407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b1b0-d739-4163-9c6b-5118f24bfd3b None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:50.497741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b1b0-d739-4163-9c6b-5118f24bfd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:50.498017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b1b0-d739-4163-9c6b-5118f24bfd3b None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda/users/0784168464c343a69d8866276838aefd/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:50.500207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566b1b0-d739-4163-9c6b-5118f24bfd3b None admin] RBAC: Authorizing `identity:create_grant(project_id=e443cfb7bf354891a724342b87c62cda, user_id=0784168464c343a69d8866276838aefd, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:50.501791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566b1b0-d739-4163-9c6b-5118f24bfd3b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:50.525682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 135/264] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:49:50 2022] PUT /identity/v3/projects/e443cfb7bf354891a724342b87c62cda/users/0784168464c343a69d8866276838aefd/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:49:50.525682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566b1b0-d739-4163-9c6b-5118f24bfd3b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:50.571805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f30073-bf29-48bf-b09a-06a95b7b4c24 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:50.653063 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566b1b0-d739-4163-9c6b-5118f24bfd3b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:50.695028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3374fb8-1fe8-4a5b-a42a-0d4ab452f290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:50.696856 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3374fb8-1fe8-4a5b-a42a-0d4ab452f290 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:50.697788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3374fb8-1fe8-4a5b-a42a-0d4ab452f290 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:50.698501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3374fb8-1fe8-4a5b-a42a-0d4ab452f290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:50.698763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ec728-3992-4178-922d-f3ec62ebe245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:50.699896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ec728-3992-4178-922d-f3ec62ebe245 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:50.700299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ec728-3992-4178-922d-f3ec62ebe245 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:50.700694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ec728-3992-4178-922d-f3ec62ebe245 None admin] PATH_INFO: `/v3/users/9dca89bc1e9548409a1ee36e5d598b8b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:50.701357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3374fb8-1fe8-4a5b-a42a-0d4ab452f290 None admin] RBAC: Adding query filter params (role.id=794ff5f5032d40a98e67fb0df718b75c, scope.project.id=e443cfb7bf354891a724342b87c62cda, user.id=f20e53c4da9242bf8b0f2d0e1e119553) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:50.702276 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3374fb8-1fe8-4a5b-a42a-0d4ab452f290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:50.705522 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3374fb8-1fe8-4a5b-a42a-0d4ab452f290 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:50.725699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ec728-3992-4178-922d-f3ec62ebe245 None admin] RBAC: Authorizing `identity:get_user(user_id=9dca89bc1e9548409a1ee36e5d598b8b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:50.725699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ec728-3992-4178-922d-f3ec62ebe245 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:50.725699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 136/265] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:50 2022] GET /identity/v3/users/9dca89bc1e9548409a1ee36e5d598b8b => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:50.737322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ec728-3992-4178-922d-f3ec62ebe245 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:50.745817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 130/266] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:50 2022] GET /identity/v3/role_assignments?user.id=f20e53c4da9242bf8b0f2d0e1e119553&scope.project.id=e443cfb7bf354891a724342b87c62cda&role.id=794ff5f5032d40a98e67fb0df718b75c => generated 608 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:50.763314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:50.921986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ec728-3992-4178-922d-f3ec62ebe245 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:50.968125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818c3ee-7b95-4cee-b8a5-f17558338a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:50.969119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818c3ee-7b95-4cee-b8a5-f17558338a92 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:50.969119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818c3ee-7b95-4cee-b8a5-f17558338a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:50.969395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818c3ee-7b95-4cee-b8a5-f17558338a92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:51.065128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818c3ee-7b95-4cee-b8a5-f17558338a92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:51.066763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818c3ee-7b95-4cee-b8a5-f17558338a92 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:51.081435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:51.187813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 137/267] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:51.194214 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3374fb8-1fe8-4a5b-a42a-0d4ab452f290 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:51.227343 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:51.323281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3374fb8-1fe8-4a5b-a42a-0d4ab452f290 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:51.378675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60ea14-0d22-4bce-a849-929b6f209a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:51.380456 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60ea14-0d22-4bce-a849-929b6f209a22 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:51.380906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60ea14-0d22-4bce-a849-929b6f209a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:51.381360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60ea14-0d22-4bce-a849-929b6f209a22 None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:51.383646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60ea14-0d22-4bce-a849-929b6f209a22 None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:51.386738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60ea14-0d22-4bce-a849-929b6f209a22 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:51.390536 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 131/268] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:51 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:51.396650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818c3ee-7b95-4cee-b8a5-f17558338a92 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:51.525068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818c3ee-7b95-4cee-b8a5-f17558338a92 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:51.538962 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:51.566125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197a82e-f84e-4da9-ac71-895c9985f92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:51.566804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197a82e-f84e-4da9-ac71-895c9985f92c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:51.567081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197a82e-f84e-4da9-ac71-895c9985f92c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:51.567523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197a82e-f84e-4da9-ac71-895c9985f92c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:51.568522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197a82e-f84e-4da9-ac71-895c9985f92c None admin] RBAC: Adding query filter params (role.id=48f409e647e64284817bad2b8ec30bf2, scope.project.id=e443cfb7bf354891a724342b87c62cda, user.id=9dca89bc1e9548409a1ee36e5d598b8b) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:51.568984 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197a82e-f84e-4da9-ac71-895c9985f92c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:51.570724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197a82e-f84e-4da9-ac71-895c9985f92c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:51.577710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 138/269] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:51 2022] GET /identity/v3/role_assignments?user.id=9dca89bc1e9548409a1ee36e5d598b8b&scope.project.id=e443cfb7bf354891a724342b87c62cda&role.id=48f409e647e64284817bad2b8ec30bf2 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:52.849917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5910-de42-4a45-8d37-8585ea421c1a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:52.850705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5910-de42-4a45-8d37-8585ea421c1a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:52.850705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5910-de42-4a45-8d37-8585ea421c1a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:52.852245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 132/270] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:52.858297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de2135-e851-43d9-8f82-6bc7258b9099 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:52.859267 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de2135-e851-43d9-8f82-6bc7258b9099 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:52.859749 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de2135-e851-43d9-8f82-6bc7258b9099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:52.880733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-59de2135-e851-43d9-8f82-6bc7258b9099 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:52.882474 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-59de2135-e851-43d9-8f82-6bc7258b9099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:53.011963 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 139/271] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:53.052678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b5910-de42-4a45-8d37-8585ea421c1a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:53.244947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:53.264498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b5910-de42-4a45-8d37-8585ea421c1a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:53.329516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae10d0d-6e1c-43f9-ae48-415c0be48e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:53.330956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae10d0d-6e1c-43f9-ae48-415c0be48e5f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:53.331437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae10d0d-6e1c-43f9-ae48-415c0be48e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:53.331938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae10d0d-6e1c-43f9-ae48-415c0be48e5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:53.352740 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae10d0d-6e1c-43f9-ae48-415c0be48e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-cae10d0d-6e1c-43f9-ae48-415c0be48e5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:53.368556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:53.369746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:53.370797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 133/272] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:53.380428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de2135-e851-43d9-8f82-6bc7258b9099 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:53.440008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf6917-cfd3-4682-9ed2-43fb841c5d0c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:53.441127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf6917-cfd3-4682-9ed2-43fb841c5d0c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:53.443057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf6917-cfd3-4682-9ed2-43fb841c5d0c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:53.445956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 134/273] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:49:53.453799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b95293-6255-48f3-aec3-a5662de2a407 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:53.454435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b95293-6255-48f3-aec3-a5662de2a407 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:53.454963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b95293-6255-48f3-aec3-a5662de2a407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:53.474136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f5b95293-6255-48f3-aec3-a5662de2a407 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:53.475781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f5b95293-6255-48f3-aec3-a5662de2a407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:53.566014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 135/274] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:53.585578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b95293-6255-48f3-aec3-a5662de2a407 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:53.592336 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de2135-e851-43d9-8f82-6bc7258b9099 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:53.597948 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:53.657300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8012496-c3a0-48db-8497-eb9e2d572852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:53.658397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8012496-c3a0-48db-8497-eb9e2d572852 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:53.658887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8012496-c3a0-48db-8497-eb9e2d572852 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:53.659421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8012496-c3a0-48db-8497-eb9e2d572852 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:53.660464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8012496-c3a0-48db-8497-eb9e2d572852 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:53.660980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8012496-c3a0-48db-8497-eb9e2d572852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:53.662757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8012496-c3a0-48db-8497-eb9e2d572852 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:53.674270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 140/275] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:53.686647 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8012496-c3a0-48db-8497-eb9e2d572852 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:53.758046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:53.793020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b95293-6255-48f3-aec3-a5662de2a407 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:53.843377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e91d-2a4b-4f43-bdf9-39493f445a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:53.844584 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e91d-2a4b-4f43-bdf9-39493f445a04 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:53.845020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e91d-2a4b-4f43-bdf9-39493f445a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:53.845496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e91d-2a4b-4f43-bdf9-39493f445a04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:53.865443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5e91d-2a4b-4f43-bdf9-39493f445a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d6b5e91d-2a4b-4f43-bdf9-39493f445a04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:53.884405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:53.885547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:53.886424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:53.886424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:53.886424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:53.886424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:53.886424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:53.886424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:53.886424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:53.886424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:53.886424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:53.886424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:53.886424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:53.886424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:53.887469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 136/276] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:49:53.894805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e91d-2a4b-4f43-bdf9-39493f445a04 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:53.904605 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8012496-c3a0-48db-8497-eb9e2d572852 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:53.958143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ed4b5-6605-4b81-a2ef-627107a77a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:53.959244 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ed4b5-6605-4b81-a2ef-627107a77a52 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:53.959624 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ed4b5-6605-4b81-a2ef-627107a77a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:53.960114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ed4b5-6605-4b81-a2ef-627107a77a52 None admin] PATH_INFO: `/v3/users/3c8623745e164560b2a37056334a14a2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:53.977583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ed4b5-6605-4b81-a2ef-627107a77a52 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8623745e164560b2a37056334a14a2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:53.979637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ed4b5-6605-4b81-a2ef-627107a77a52 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:53.982247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 141/277] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:53 2022] GET /identity/v3/users/3c8623745e164560b2a37056334a14a2 => generated 293 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:53.984640 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d5df8-a1b0-4cc8-89a2-b1dca6418ff3 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:53.985241 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d5df8-a1b0-4cc8-89a2-b1dca6418ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:53.985849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d5df8-a1b0-4cc8-89a2-b1dca6418ff3 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:53.993735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 142/278] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:53.995188 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d5df8-a1b0-4cc8-89a2-b1dca6418ff3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:54.028325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:54.036305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e91d-2a4b-4f43-bdf9-39493f445a04 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:54.081073 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c52cbc-e2ab-4d97-92bd-aa2cfbca1be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:54.082355 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c52cbc-e2ab-4d97-92bd-aa2cfbca1be0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:54.082821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c52cbc-e2ab-4d97-92bd-aa2cfbca1be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:54.083307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c52cbc-e2ab-4d97-92bd-aa2cfbca1be0 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:54.084757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c52cbc-e2ab-4d97-92bd-aa2cfbca1be0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:54.085322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c52cbc-e2ab-4d97-92bd-aa2cfbca1be0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:54.088347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c52cbc-e2ab-4d97-92bd-aa2cfbca1be0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:54.104356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 137/279] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:54.107118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3442cd6-c3fc-4ac6-8b7b-a1237c9ec677 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:54.107118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3442cd6-c3fc-4ac6-8b7b-a1237c9ec677 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:54.107467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3442cd6-c3fc-4ac6-8b7b-a1237c9ec677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:54.115820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:54.128657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d5df8-a1b0-4cc8-89a2-b1dca6418ff3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:54.138172 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f3442cd6-c3fc-4ac6-8b7b-a1237c9ec677 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:54.140224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f3442cd6-c3fc-4ac6-8b7b-a1237c9ec677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:54.165892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b231536-c971-4955-b59f-aa44964c519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:54.166576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b231536-c971-4955-b59f-aa44964c519d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:54.166864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b231536-c971-4955-b59f-aa44964c519d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:54.167140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b231536-c971-4955-b59f-aa44964c519d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:54.167990 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b231536-c971-4955-b59f-aa44964c519d None admin] RBAC: Adding query filter params (role.id=3edce31396af478ea1ea7b76d33da530, user.id=3c8623745e164560b2a37056334a14a2, scope.system=all) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:54.168306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b231536-c971-4955-b59f-aa44964c519d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:54.169817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b231536-c971-4955-b59f-aa44964c519d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:54.176036 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 143/280] 10.222.0.43 () {56 vars in 1308 bytes} [Mon Jul 4 07:49:53 2022] GET /identity/v3/role_assignments?user.id=3c8623745e164560b2a37056334a14a2&scope.system=all&role.id=3edce31396af478ea1ea7b76d33da530 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:54.177154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b231536-c971-4955-b59f-aa44964c519d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:54.281797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 138/281] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:54.288463 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b108d-e0a9-4416-aa96-5e03986e65fa None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:54.288965 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b108d-e0a9-4416-aa96-5e03986e65fa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:54.289451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b108d-e0a9-4416-aa96-5e03986e65fa None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:54.292513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 139/282] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:54.301729 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8b026-a7e6-4bcd-a02b-d8a611307929 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:54.302233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8b026-a7e6-4bcd-a02b-d8a611307929 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:54.302728 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8b026-a7e6-4bcd-a02b-d8a611307929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:54.333556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:54.340340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d1f8b026-a7e6-4bcd-a02b-d8a611307929 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:54.342397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d1f8b026-a7e6-4bcd-a02b-d8a611307929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:54.348702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b231536-c971-4955-b59f-aa44964c519d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:54.400504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da76c9-d92b-480f-a73d-6fc0235d9c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:54.401447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da76c9-d92b-480f-a73d-6fc0235d9c76 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:54.401765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da76c9-d92b-480f-a73d-6fc0235d9c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:54.402048 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da76c9-d92b-480f-a73d-6fc0235d9c76 None admin] PATH_INFO: `/v3/users/0784168464c343a69d8866276838aefd` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:54.403366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da76c9-d92b-480f-a73d-6fc0235d9c76 None admin] RBAC: Authorizing `identity:get_user(user_id=0784168464c343a69d8866276838aefd)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:54.404880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da76c9-d92b-480f-a73d-6fc0235d9c76 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:54.406673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 144/283] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:54 2022] GET /identity/v3/users/0784168464c343a69d8866276838aefd => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:54.413111 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da76c9-d92b-480f-a73d-6fc0235d9c76 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:54.454329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 140/284] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:49:54.455715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8b026-a7e6-4bcd-a02b-d8a611307929 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:54.556453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:54.565173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da76c9-d92b-480f-a73d-6fc0235d9c76 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:54.609092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ed3f3-ee68-47a1-a8fe-0921874a8921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:54.610550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ed3f3-ee68-47a1-a8fe-0921874a8921 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:54.611030 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ed3f3-ee68-47a1-a8fe-0921874a8921 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:54.611821 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ed3f3-ee68-47a1-a8fe-0921874a8921 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:54.613881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ed3f3-ee68-47a1-a8fe-0921874a8921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:54.616295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ed3f3-ee68-47a1-a8fe-0921874a8921 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:54.616415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8b026-a7e6-4bcd-a02b-d8a611307929 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:54.624519 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 145/285] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:54.624519 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ed3f3-ee68-47a1-a8fe-0921874a8921 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:54.666020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:54.689465 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617495f-fcca-40fa-9697-c95ec344ba59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:54.690606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617495f-fcca-40fa-9697-c95ec344ba59 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:54.691109 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617495f-fcca-40fa-9697-c95ec344ba59 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:54.691607 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617495f-fcca-40fa-9697-c95ec344ba59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:54.793955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ed3f3-ee68-47a1-a8fe-0921874a8921 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:54.805442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617495f-fcca-40fa-9697-c95ec344ba59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:54.808511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617495f-fcca-40fa-9697-c95ec344ba59 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:54.840093 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8098f4e-c601-48d5-91f8-6e2f95486c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:54.841394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8098f4e-c601-48d5-91f8-6e2f95486c78 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:54.841882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8098f4e-c601-48d5-91f8-6e2f95486c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:54.842637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8098f4e-c601-48d5-91f8-6e2f95486c78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:54.858258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8098f4e-c601-48d5-91f8-6e2f95486c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c8098f4e-c601-48d5-91f8-6e2f95486c78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:54.873349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:54.874595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:54.875447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 146/286] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:54.878079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8098f4e-c601-48d5-91f8-6e2f95486c78 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:54.922494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:54.990118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 141/287] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:49:54.991417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617495f-fcca-40fa-9697-c95ec344ba59 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:55.040386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:55.069603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8098f4e-c601-48d5-91f8-6e2f95486c78 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:55.137740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7dfbf5-b513-4ca3-8498-54320ddc5aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:55.138853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7dfbf5-b513-4ca3-8498-54320ddc5aee None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:55.139293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7dfbf5-b513-4ca3-8498-54320ddc5aee None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:55.139724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7dfbf5-b513-4ca3-8498-54320ddc5aee None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:55.140932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7dfbf5-b513-4ca3-8498-54320ddc5aee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:55.144154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7dfbf5-b513-4ca3-8498-54320ddc5aee None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:55.176910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 147/288] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:49:54 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:49:55.191056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617495f-fcca-40fa-9697-c95ec344ba59 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:55.191360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7dfbf5-b513-4ca3-8498-54320ddc5aee None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:55.274520 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202ea94-96f0-42e4-90dc-e8abc0669c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:55.275682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202ea94-96f0-42e4-90dc-e8abc0669c2c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:55.276154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202ea94-96f0-42e4-90dc-e8abc0669c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:55.276625 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202ea94-96f0-42e4-90dc-e8abc0669c2c None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:55.277916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202ea94-96f0-42e4-90dc-e8abc0669c2c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:55.278442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202ea94-96f0-42e4-90dc-e8abc0669c2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:55.280862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202ea94-96f0-42e4-90dc-e8abc0669c2c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:55.306464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 142/289] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:55.309688 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202ea94-96f0-42e4-90dc-e8abc0669c2c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:55.362826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:55.422859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7dfbf5-b513-4ca3-8498-54320ddc5aee None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:55.482641 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ab7f-6372-403a-b4d0-871476ff711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:55.483782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ab7f-6372-403a-b4d0-871476ff711f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:55.484068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ab7f-6372-403a-b4d0-871476ff711f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:55.484348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ab7f-6372-403a-b4d0-871476ff711f None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:55.492482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4ab7f-6372-403a-b4d0-871476ff711f None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:55.493505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4ab7f-6372-403a-b4d0-871476ff711f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:55.495981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 148/290] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:55 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:55.504237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ab7f-6372-403a-b4d0-871476ff711f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:55.505422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202ea94-96f0-42e4-90dc-e8abc0669c2c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:55.520787 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:55.562301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e685432-c6e4-491e-97d8-96e90f59f132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:55.564001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e685432-c6e4-491e-97d8-96e90f59f132 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:55.564314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e685432-c6e4-491e-97d8-96e90f59f132 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:55.564618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e685432-c6e4-491e-97d8-96e90f59f132 None admin] PATH_INFO: `/v3/users/9dca89bc1e9548409a1ee36e5d598b8b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:55.566226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e685432-c6e4-491e-97d8-96e90f59f132 None admin] RBAC: Authorizing `identity:get_user(user_id=9dca89bc1e9548409a1ee36e5d598b8b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:55.568393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e685432-c6e4-491e-97d8-96e90f59f132 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:55.571780 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 143/291] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:55 2022] GET /identity/v3/users/9dca89bc1e9548409a1ee36e5d598b8b => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:55.577695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e685432-c6e4-491e-97d8-96e90f59f132 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:55.673125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ab7f-6372-403a-b4d0-871476ff711f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:55.713658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b02d2d-33ee-455c-b56c-4a7827246e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:55.714362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b02d2d-33ee-455c-b56c-4a7827246e85 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:55.714643 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b02d2d-33ee-455c-b56c-4a7827246e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:55.714917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b02d2d-33ee-455c-b56c-4a7827246e85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:55.715933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b02d2d-33ee-455c-b56c-4a7827246e85 None admin] RBAC: Adding query filter params (role.id=3edce31396af478ea1ea7b76d33da530, scope.project.id=e443cfb7bf354891a724342b87c62cda, user.id=0784168464c343a69d8866276838aefd) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:55.716275 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b02d2d-33ee-455c-b56c-4a7827246e85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:55.718734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b02d2d-33ee-455c-b56c-4a7827246e85 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:55.721998 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:55.727573 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 149/292] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:49:55 2022] GET /identity/v3/role_assignments?user.id=0784168464c343a69d8866276838aefd&scope.project.id=e443cfb7bf354891a724342b87c62cda&role.id=3edce31396af478ea1ea7b76d33da530 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:55.752163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e685432-c6e4-491e-97d8-96e90f59f132 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:55.800995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56fc8bc-1a34-4978-ae7a-1fdd558dc3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:55.801957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fc8bc-1a34-4978-ae7a-1fdd558dc3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:55.802238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fc8bc-1a34-4978-ae7a-1fdd558dc3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:55.802510 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fc8bc-1a34-4978-ae7a-1fdd558dc3f0 None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:55.803920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56fc8bc-1a34-4978-ae7a-1fdd558dc3f0 None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:55.805588 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56fc8bc-1a34-4978-ae7a-1fdd558dc3f0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:55.808240 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 144/293] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:49:55 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:55.814295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b02d2d-33ee-455c-b56c-4a7827246e85 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:55.948680 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:55.993468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b02d2d-33ee-455c-b56c-4a7827246e85 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:56.030360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9eb06a-788e-4230-a406-cb1705247ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:56.031279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9eb06a-788e-4230-a406-cb1705247ea1 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:56.031566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9eb06a-788e-4230-a406-cb1705247ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:56.031842 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9eb06a-788e-4230-a406-cb1705247ea1 None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda/users/9dca89bc1e9548409a1ee36e5d598b8b/roles/48f409e647e64284817bad2b8ec30bf2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:56.033948 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9eb06a-788e-4230-a406-cb1705247ea1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e443cfb7bf354891a724342b87c62cda, user_id=9dca89bc1e9548409a1ee36e5d598b8b, role_id=48f409e647e64284817bad2b8ec30bf2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:56.035436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9eb06a-788e-4230-a406-cb1705247ea1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:56.053675 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 150/294] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:49:55 2022] PUT /identity/v3/projects/e443cfb7bf354891a724342b87c62cda/users/9dca89bc1e9548409a1ee36e5d598b8b/roles/48f409e647e64284817bad2b8ec30bf2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:49:56.971019 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530679e-3bb1-452a-a1b3-677e08a383d7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:56.971617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530679e-3bb1-452a-a1b3-677e08a383d7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:56.971847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530679e-3bb1-452a-a1b3-677e08a383d7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:56.974135 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 145/295] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:56.982865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec94c-f09a-42cc-bc00-00e0341cf895 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:56.983302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec94c-f09a-42cc-bc00-00e0341cf895 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:56.983730 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec94c-f09a-42cc-bc00-00e0341cf895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:57.010026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e85ec94c-f09a-42cc-bc00-00e0341cf895 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:57.011841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e85ec94c-f09a-42cc-bc00-00e0341cf895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:57.121230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 151/296] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:57.160231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530679e-3bb1-452a-a1b3-677e08a383d7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:57.318235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530679e-3bb1-452a-a1b3-677e08a383d7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:57.357482 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8139fa-89a4-45c0-b7ea-09c65adeac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:57.358345 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8139fa-89a4-45c0-b7ea-09c65adeac9b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:57.358633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8139fa-89a4-45c0-b7ea-09c65adeac9b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:57.358909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8139fa-89a4-45c0-b7ea-09c65adeac9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:57.371749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8139fa-89a4-45c0-b7ea-09c65adeac9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9d8139fa-89a4-45c0-b7ea-09c65adeac9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:57.380151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:57.382494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:57.383974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 146/297] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:57.388563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ec94c-f09a-42cc-bc00-00e0341cf895 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:57.460796 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:57.508912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ec94c-f09a-42cc-bc00-00e0341cf895 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:57.551075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f1161-0e74-4d4e-9b19-75605df81f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:57.551779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f1161-0e74-4d4e-9b19-75605df81f0e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:57.552059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f1161-0e74-4d4e-9b19-75605df81f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:57.552310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f1161-0e74-4d4e-9b19-75605df81f0e None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:57.553076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f1161-0e74-4d4e-9b19-75605df81f0e None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:57.553383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f1161-0e74-4d4e-9b19-75605df81f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:57.554883 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f1161-0e74-4d4e-9b19-75605df81f0e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:57.569662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 152/298] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:57.573908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8139fa-89a4-45c0-b7ea-09c65adeac9b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:57.598825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:57.694207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8139fa-89a4-45c0-b7ea-09c65adeac9b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:57.732474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c864fb-834f-4958-b3a8-0d2e0d7cc4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:57.733379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c864fb-834f-4958-b3a8-0d2e0d7cc4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:57.733677 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c864fb-834f-4958-b3a8-0d2e0d7cc4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:57.733956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c864fb-834f-4958-b3a8-0d2e0d7cc4e2 None admin] PATH_INFO: `/v3/users/3c8623745e164560b2a37056334a14a2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:57.735221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c864fb-834f-4958-b3a8-0d2e0d7cc4e2 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8623745e164560b2a37056334a14a2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:57.736700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c864fb-834f-4958-b3a8-0d2e0d7cc4e2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:57.738831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 147/299] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:57 2022] GET /identity/v3/users/3c8623745e164560b2a37056334a14a2 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:57.744869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f1161-0e74-4d4e-9b19-75605df81f0e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:57.861884 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f1161-0e74-4d4e-9b19-75605df81f0e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:57.894281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:57.899783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce2e8b-4be8-4330-a306-91a9b1a1e248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:57.900723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce2e8b-4be8-4330-a306-91a9b1a1e248 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:57.901037 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce2e8b-4be8-4330-a306-91a9b1a1e248 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:57.901319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce2e8b-4be8-4330-a306-91a9b1a1e248 None admin] PATH_INFO: `/v3/system/users/3c8623745e164560b2a37056334a14a2/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:57.903051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce2e8b-4be8-4330-a306-91a9b1a1e248 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3c8623745e164560b2a37056334a14a2, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:57.904761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce2e8b-4be8-4330-a306-91a9b1a1e248 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:57.918288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 153/300] 10.222.0.43 () {58 vars in 1334 bytes} [Mon Jul 4 07:49:57 2022] PUT /identity/v3/system/users/3c8623745e164560b2a37056334a14a2/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:49:58.012088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044079-936e-47d4-b295-136da2380ef4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:58.012753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044079-936e-47d4-b295-136da2380ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:58.012753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044079-936e-47d4-b295-136da2380ef4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:58.014540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 148/301] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:58.020731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afecd9e-b76e-421d-b8a0-ed27c7c06f97 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:58.021041 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afecd9e-b76e-421d-b8a0-ed27c7c06f97 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:58.021289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afecd9e-b76e-421d-b8a0-ed27c7c06f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:58.041327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3afecd9e-b76e-421d-b8a0-ed27c7c06f97 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:58.042751 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3afecd9e-b76e-421d-b8a0-ed27c7c06f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:58.130420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 154/302] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:58.135391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c9123-f006-49dc-b898-0e8fafefe963 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:58.135974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c9123-f006-49dc-b898-0e8fafefe963 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:58.136492 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c9123-f006-49dc-b898-0e8fafefe963 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:58.138183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 149/303] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:58.144039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c5fda-21f0-4263-8961-94535351017e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:58.144332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c5fda-21f0-4263-8961-94535351017e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:58.144598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c5fda-21f0-4263-8961-94535351017e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:58.150882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c9123-f006-49dc-b898-0e8fafefe963 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:58.165907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-397c5fda-21f0-4263-8961-94535351017e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:58.167379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-397c5fda-21f0-4263-8961-94535351017e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:58.259700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:58.262995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 155/304] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:58.279430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c9123-f006-49dc-b898-0e8fafefe963 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:58.287263 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c5fda-21f0-4263-8961-94535351017e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:58.322097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ea18-d59d-407f-b908-c5b96cdb7ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:58.322957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ea18-d59d-407f-b908-c5b96cdb7ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:58.323238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ea18-d59d-407f-b908-c5b96cdb7ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:58.323511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ea18-d59d-407f-b908-c5b96cdb7ba1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:58.324788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854ea18-d59d-407f-b908-c5b96cdb7ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:58.326331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854ea18-d59d-407f-b908-c5b96cdb7ba1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:58.328403 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 150/305] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:49:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:58.335165 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ea18-d59d-407f-b908-c5b96cdb7ba1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:58.354698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:58.415908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c5fda-21f0-4263-8961-94535351017e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:58.461820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b44830-a12d-406d-9d37-7cfa720a9f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:58.462679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b44830-a12d-406d-9d37-7cfa720a9f22 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:58.462969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b44830-a12d-406d-9d37-7cfa720a9f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:58.463246 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b44830-a12d-406d-9d37-7cfa720a9f22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:58.466017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ea18-d59d-407f-b908-c5b96cdb7ba1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:58.476472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b44830-a12d-406d-9d37-7cfa720a9f22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e5b44830-a12d-406d-9d37-7cfa720a9f22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:58.485071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:58.486092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:49:58.486995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 156/306] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:49:58.492901 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b44830-a12d-406d-9d37-7cfa720a9f22 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:58.513467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31c38-c6b3-45f5-9c84-8533bfad94d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:58.514637 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31c38-c6b3-45f5-9c84-8533bfad94d4 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:58.515286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31c38-c6b3-45f5-9c84-8533bfad94d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:58.515774 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31c38-c6b3-45f5-9c84-8533bfad94d4 None admin] PATH_INFO: `/v3/groups` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:58.516871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e31c38-c6b3-45f5-9c84-8533bfad94d4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:58.518800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e31c38-c6b3-45f5-9c84-8533bfad94d4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:58.546882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 151/307] 10.222.0.43 () {60 vars in 1134 bytes} [Mon Jul 4 07:49:58 2022] POST /identity/v3/groups => generated 229 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:49:58.629357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b44830-a12d-406d-9d37-7cfa720a9f22 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:58.641636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:58.673520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60baf82-b412-4caa-a6e3-b0c6446ec90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:58.674168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60baf82-b412-4caa-a6e3-b0c6446ec90c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:58.674456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60baf82-b412-4caa-a6e3-b0c6446ec90c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:58.674734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60baf82-b412-4caa-a6e3-b0c6446ec90c None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:58.675593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60baf82-b412-4caa-a6e3-b0c6446ec90c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:58.675908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60baf82-b412-4caa-a6e3-b0c6446ec90c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:58.677367 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60baf82-b412-4caa-a6e3-b0c6446ec90c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:58.692459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 157/308] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:49:58.694298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31c38-c6b3-45f5-9c84-8533bfad94d4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:58.716590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:58.854138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31c38-c6b3-45f5-9c84-8533bfad94d4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:58.896203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db987df-7700-439f-b3e5-f3ee4442f943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:58.897666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db987df-7700-439f-b3e5-f3ee4442f943 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:58.898213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db987df-7700-439f-b3e5-f3ee4442f943 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:58.898729 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db987df-7700-439f-b3e5-f3ee4442f943 None admin] PATH_INFO: `/v3/users/429aaa62ee4c4faeae0637a870cb0355` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:58.914532 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db987df-7700-439f-b3e5-f3ee4442f943 None admin] RBAC: Authorizing `identity:get_user(user_id=429aaa62ee4c4faeae0637a870cb0355)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:58.916392 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db987df-7700-439f-b3e5-f3ee4442f943 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:58.918763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 152/309] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:58 2022] GET /identity/v3/users/429aaa62ee4c4faeae0637a870cb0355 => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:58.927508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60baf82-b412-4caa-a6e3-b0c6446ec90c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:59.035126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60baf82-b412-4caa-a6e3-b0c6446ec90c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:59.072678 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54162a80-c75f-40a8-a85c-a39d828e45e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:59.073362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54162a80-c75f-40a8-a85c-a39d828e45e0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:59.073671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54162a80-c75f-40a8-a85c-a39d828e45e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:59.073955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54162a80-c75f-40a8-a85c-a39d828e45e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:59.074807 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54162a80-c75f-40a8-a85c-a39d828e45e0 None admin] RBAC: Adding query filter params (role.id=48f409e647e64284817bad2b8ec30bf2, user.id=429aaa62ee4c4faeae0637a870cb0355, scope.system=all) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:59.075120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54162a80-c75f-40a8-a85c-a39d828e45e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:59.076593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54162a80-c75f-40a8-a85c-a39d828e45e0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:59.089806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 158/310] 10.222.0.43 () {56 vars in 1308 bytes} [Mon Jul 4 07:49:58 2022] GET /identity/v3/role_assignments?user.id=429aaa62ee4c4faeae0637a870cb0355&scope.system=all&role.id=48f409e647e64284817bad2b8ec30bf2 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:59.304734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528970f-763f-48ba-be24-74056876bd81 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:59.305304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528970f-763f-48ba-be24-74056876bd81 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:59.305304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528970f-763f-48ba-be24-74056876bd81 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:59.306935 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 153/311] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:49:59.314023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebc18c-b600-42b6-819a-be00e6d655a1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:59.314529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebc18c-b600-42b6-819a-be00e6d655a1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:59.314799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebc18c-b600-42b6-819a-be00e6d655a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:59.333072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-42ebc18c-b600-42b6-819a-be00e6d655a1 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:49:59.334342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-42ebc18c-b600-42b6-819a-be00e6d655a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:49:59.363674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:59.409645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 159/312] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:49:59.433489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528970f-763f-48ba-be24-74056876bd81 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:59.519562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:49:59.568749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528970f-763f-48ba-be24-74056876bd81 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:59.608112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94955c51-905e-4802-925b-8950cb63ac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:59.608959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94955c51-905e-4802-925b-8950cb63ac3f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:59.609241 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94955c51-905e-4802-925b-8950cb63ac3f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:59.609516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94955c51-905e-4802-925b-8950cb63ac3f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:59.622103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94955c51-905e-4802-925b-8950cb63ac3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-94955c51-905e-4802-925b-8950cb63ac3f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:49:59.630655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:49:59.631737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:49:59.632634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 154/313] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:49:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:49:59.638456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebc18c-b600-42b6-819a-be00e6d655a1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:59.745033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebc18c-b600-42b6-819a-be00e6d655a1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:59.782310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff705d5-0fb3-4bc0-8ca1-368995ee15f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:59.783007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff705d5-0fb3-4bc0-8ca1-368995ee15f5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:59.783291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff705d5-0fb3-4bc0-8ca1-368995ee15f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:59.783568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff705d5-0fb3-4bc0-8ca1-368995ee15f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:59.784331 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff705d5-0fb3-4bc0-8ca1-368995ee15f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:49:59.784640 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff705d5-0fb3-4bc0-8ca1-368995ee15f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:59.786076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff705d5-0fb3-4bc0-8ca1-368995ee15f5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:59.796091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 160/314] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:49:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:59.803062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94955c51-905e-4802-925b-8950cb63ac3f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:49:59.925604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94955c51-905e-4802-925b-8950cb63ac3f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:49:59.970577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab519-8fc1-4c55-8967-ecd7d59e6543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:49:59.971760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab519-8fc1-4c55-8967-ecd7d59e6543 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:49:59.972184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab519-8fc1-4c55-8967-ecd7d59e6543 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:49:59.972591 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab519-8fc1-4c55-8967-ecd7d59e6543 None admin] PATH_INFO: `/v3/users/9dca89bc1e9548409a1ee36e5d598b8b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:49:59.974045 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ab519-8fc1-4c55-8967-ecd7d59e6543 None admin] RBAC: Authorizing `identity:get_user(user_id=9dca89bc1e9548409a1ee36e5d598b8b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:49:59.975708 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ab519-8fc1-4c55-8967-ecd7d59e6543 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:49:59.978370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 155/315] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:49:59 2022] GET /identity/v3/users/9dca89bc1e9548409a1ee36e5d598b8b => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:49:59.989648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff705d5-0fb3-4bc0-8ca1-368995ee15f5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:00.015613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:00.105724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff705d5-0fb3-4bc0-8ca1-368995ee15f5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:00.143779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191db6af-d611-42e5-b441-6d40765f0283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:00.144546 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191db6af-d611-42e5-b441-6d40765f0283 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:00.144834 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191db6af-d611-42e5-b441-6d40765f0283 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:00.145116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191db6af-d611-42e5-b441-6d40765f0283 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:00.225438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191db6af-d611-42e5-b441-6d40765f0283 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:00.227110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191db6af-d611-42e5-b441-6d40765f0283 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:00.257143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:00.338994 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 161/316] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:49:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:00.344349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab519-8fc1-4c55-8967-ecd7d59e6543 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:00.347895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:00.522548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab519-8fc1-4c55-8967-ecd7d59e6543 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:00.583636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce415891-f62d-4a05-9dea-f0e06096fb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:00.585586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce415891-f62d-4a05-9dea-f0e06096fb2d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:00.587402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce415891-f62d-4a05-9dea-f0e06096fb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:00.588098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce415891-f62d-4a05-9dea-f0e06096fb2d None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:00.590666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce415891-f62d-4a05-9dea-f0e06096fb2d None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:00.593747 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce415891-f62d-4a05-9dea-f0e06096fb2d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:00.601751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 156/317] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:00 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:00.607965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191db6af-d611-42e5-b441-6d40765f0283 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:00.726070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40d8e6-b7bf-41c7-a6f9-7b84dd33124d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:00.726747 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40d8e6-b7bf-41c7-a6f9-7b84dd33124d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:00.727194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40d8e6-b7bf-41c7-a6f9-7b84dd33124d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:00.730341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 157/318] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:00.739570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69086293-e9b1-49a1-b739-88ac0ca3470c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:00.740082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69086293-e9b1-49a1-b739-88ac0ca3470c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:00.740594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69086293-e9b1-49a1-b739-88ac0ca3470c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:00.773662 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-69086293-e9b1-49a1-b739-88ac0ca3470c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:00.775920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-69086293-e9b1-49a1-b739-88ac0ca3470c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:00.817475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191db6af-d611-42e5-b441-6d40765f0283 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:00.878504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:00.897658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d30b4f-1f5e-4758-85fe-3baf8aab0188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:00.899240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d30b4f-1f5e-4758-85fe-3baf8aab0188 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:00.899973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d30b4f-1f5e-4758-85fe-3baf8aab0188 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:00.900643 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d30b4f-1f5e-4758-85fe-3baf8aab0188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:00.902598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d30b4f-1f5e-4758-85fe-3baf8aab0188 None admin] RBAC: Adding query filter params (role.id=48f409e647e64284817bad2b8ec30bf2, scope.project.id=e443cfb7bf354891a724342b87c62cda, user.id=9dca89bc1e9548409a1ee36e5d598b8b) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:00.903441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d30b4f-1f5e-4758-85fe-3baf8aab0188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:00.906455 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d30b4f-1f5e-4758-85fe-3baf8aab0188 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:00.918468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 162/319] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:00 2022] GET /identity/v3/role_assignments?user.id=9dca89bc1e9548409a1ee36e5d598b8b&scope.project.id=e443cfb7bf354891a724342b87c62cda&role.id=48f409e647e64284817bad2b8ec30bf2 => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:00.926567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:00.937692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 158/320] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:00.982716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d30b4f-1f5e-4758-85fe-3baf8aab0188 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:01.006204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37712d46-17bb-48af-bfb6-16916b77e77a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:01.007283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37712d46-17bb-48af-bfb6-16916b77e77a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:01.008112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37712d46-17bb-48af-bfb6-16916b77e77a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:01.011931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 159/321] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:01.018641 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c74cf-f5c2-42c8-9d9c-9664c193d189 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:01.019759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c74cf-f5c2-42c8-9d9c-9664c193d189 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:01.020698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c74cf-f5c2-42c8-9d9c-9664c193d189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:01.051387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5f4c74cf-f5c2-42c8-9d9c-9664c193d189 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:01.053688 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5f4c74cf-f5c2-42c8-9d9c-9664c193d189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:01.206228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d30b4f-1f5e-4758-85fe-3baf8aab0188 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:01.211062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 160/322] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:50:01.256780 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c74cf-f5c2-42c8-9d9c-9664c193d189 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:01.280629 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a079e6d-6861-44b4-9ffb-ddaef8153208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:01.281973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a079e6d-6861-44b4-9ffb-ddaef8153208 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:01.282408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a079e6d-6861-44b4-9ffb-ddaef8153208 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:01.282853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a079e6d-6861-44b4-9ffb-ddaef8153208 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:01.301692 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a079e6d-6861-44b4-9ffb-ddaef8153208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1a079e6d-6861-44b4-9ffb-ddaef8153208 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:01.314033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:01.315983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:01.317523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 163/323] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:01.319237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0e8ad-6c36-4227-83ae-0683307a9f0b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:01.319237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0e8ad-6c36-4227-83ae-0683307a9f0b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:01.319677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0e8ad-6c36-4227-83ae-0683307a9f0b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:01.322924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 164/324] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:01.328865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0e8ad-6c36-4227-83ae-0683307a9f0b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:01.391914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:01.404323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c74cf-f5c2-42c8-9d9c-9664c193d189 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:01.444216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:01.448618 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0e8ad-6c36-4227-83ae-0683307a9f0b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:01.456372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2936a-cd0f-4492-bba6-98e3c6eb19f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:01.457197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2936a-cd0f-4492-bba6-98e3c6eb19f4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:01.457479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2936a-cd0f-4492-bba6-98e3c6eb19f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:01.457793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2936a-cd0f-4492-bba6-98e3c6eb19f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:01.459015 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2936a-cd0f-4492-bba6-98e3c6eb19f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:01.460503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2936a-cd0f-4492-bba6-98e3c6eb19f4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:01.462528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 161/325] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:01.462528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dd1a1-f792-40c4-b009-4a32cd9e1ac8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:01.462528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dd1a1-f792-40c4-b009-4a32cd9e1ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:01.462528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dd1a1-f792-40c4-b009-4a32cd9e1ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:01.490472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-013dd1a1-f792-40c4-b009-4a32cd9e1ac8 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:01.491714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-013dd1a1-f792-40c4-b009-4a32cd9e1ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:01.491862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937c8ff-eb52-4352-a609-f191455f5eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:01.493243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937c8ff-eb52-4352-a609-f191455f5eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:01.493990 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937c8ff-eb52-4352-a609-f191455f5eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:01.494349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937c8ff-eb52-4352-a609-f191455f5eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:01.495681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937c8ff-eb52-4352-a609-f191455f5eb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:01.496236 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937c8ff-eb52-4352-a609-f191455f5eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:01.498862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937c8ff-eb52-4352-a609-f191455f5eb0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:01.510836 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 165/326] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:50:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:01.510836 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937c8ff-eb52-4352-a609-f191455f5eb0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:01.570903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 162/327] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:01.571716 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013dd1a1-f792-40c4-b009-4a32cd9e1ac8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:01.645991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937c8ff-eb52-4352-a609-f191455f5eb0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:01.670969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:01.689550 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013dd1a1-f792-40c4-b009-4a32cd9e1ac8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:01.695575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4774e-2340-4e09-8f39-abea884219ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:01.696558 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4774e-2340-4e09-8f39-abea884219ef None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:01.696941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4774e-2340-4e09-8f39-abea884219ef None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:01.697312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4774e-2340-4e09-8f39-abea884219ef None admin] PATH_INFO: `/v3/groups` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:01.698195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4774e-2340-4e09-8f39-abea884219ef None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:01.699852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4774e-2340-4e09-8f39-abea884219ef None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:01.721459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 166/328] 10.222.0.43 () {60 vars in 1134 bytes} [Mon Jul 4 07:50:01 2022] POST /identity/v3/groups => generated 226 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:50:01.722899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4774e-2340-4e09-8f39-abea884219ef None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:01.734330 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:01.747642 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b027f-efb2-47e5-bc82-6f997b380d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:01.749113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b027f-efb2-47e5-bc82-6f997b380d58 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:01.749417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b027f-efb2-47e5-bc82-6f997b380d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:01.749847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b027f-efb2-47e5-bc82-6f997b380d58 None admin] PATH_INFO: `/v3/users/3c8623745e164560b2a37056334a14a2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:01.751544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b027f-efb2-47e5-bc82-6f997b380d58 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8623745e164560b2a37056334a14a2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:01.753167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b027f-efb2-47e5-bc82-6f997b380d58 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:01.755388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 163/329] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:50:01 2022] GET /identity/v3/users/3c8623745e164560b2a37056334a14a2 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:01.762435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b027f-efb2-47e5-bc82-6f997b380d58 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:01.888889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4774e-2340-4e09-8f39-abea884219ef None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:01.918332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b027f-efb2-47e5-bc82-6f997b380d58 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:01.944269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b46c1b-dfe2-44eb-91df-affc837e2340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:01.946053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b46c1b-dfe2-44eb-91df-affc837e2340 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:01.946523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b46c1b-dfe2-44eb-91df-affc837e2340 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:01.946962 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b46c1b-dfe2-44eb-91df-affc837e2340 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:01.964033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b46c1b-dfe2-44eb-91df-affc837e2340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-97b46c1b-dfe2-44eb-91df-affc837e2340 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:01.976177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:01.977946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:01.979012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:01.979012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:01.979012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:01.979012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:01.979012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:01.979012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:01.979012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:01.979012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:01.979012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:01.979012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:01.979012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:50:01.979012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:01.979787 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f90e83-9d64-47ac-a5ed-314ae0327a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:01.980014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 167/330] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:01.981391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f90e83-9d64-47ac-a5ed-314ae0327a65 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:01.982020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f90e83-9d64-47ac-a5ed-314ae0327a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:01.982533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f90e83-9d64-47ac-a5ed-314ae0327a65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:01.984247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f90e83-9d64-47ac-a5ed-314ae0327a65 None admin] RBAC: Adding query filter params (role.id=3edce31396af478ea1ea7b76d33da530, user.id=3c8623745e164560b2a37056334a14a2, scope.system=all) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:01.984671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f90e83-9d64-47ac-a5ed-314ae0327a65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:01.987289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f90e83-9d64-47ac-a5ed-314ae0327a65 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:01.989016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b46c1b-dfe2-44eb-91df-affc837e2340 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:02.013845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 164/331] 10.222.0.43 () {56 vars in 1308 bytes} [Mon Jul 4 07:50:01 2022] GET /identity/v3/role_assignments?user.id=3c8623745e164560b2a37056334a14a2&scope.system=all&role.id=3edce31396af478ea1ea7b76d33da530 => generated 510 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:02.045243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:02.191035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b46c1b-dfe2-44eb-91df-affc837e2340 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:02.255852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a38a7d-bae2-49fe-abf4-3f1a5ff2812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:02.257005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a38a7d-bae2-49fe-abf4-3f1a5ff2812b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:02.257510 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a38a7d-bae2-49fe-abf4-3f1a5ff2812b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:02.258292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a38a7d-bae2-49fe-abf4-3f1a5ff2812b None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:02.259331 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a38a7d-bae2-49fe-abf4-3f1a5ff2812b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:02.259881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a38a7d-bae2-49fe-abf4-3f1a5ff2812b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:02.261615 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a38a7d-bae2-49fe-abf4-3f1a5ff2812b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:02.273140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 168/332] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:50:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:02.283535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f90e83-9d64-47ac-a5ed-314ae0327a65 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:02.413257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:02.504656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f90e83-9d64-47ac-a5ed-314ae0327a65 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:02.569803 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a17827-92c1-4e1b-b321-966157b2add0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:02.571428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a17827-92c1-4e1b-b321-966157b2add0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:02.571919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a17827-92c1-4e1b-b321-966157b2add0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:02.572415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a17827-92c1-4e1b-b321-966157b2add0 None admin] PATH_INFO: `/v3/users/429aaa62ee4c4faeae0637a870cb0355` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:02.576790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a17827-92c1-4e1b-b321-966157b2add0 None admin] RBAC: Authorizing `identity:get_user(user_id=429aaa62ee4c4faeae0637a870cb0355)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:02.580601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a17827-92c1-4e1b-b321-966157b2add0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:02.584406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 165/333] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:50:02 2022] GET /identity/v3/users/429aaa62ee4c4faeae0637a870cb0355 => generated 293 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:02.593376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a38a7d-bae2-49fe-abf4-3f1a5ff2812b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:02.752403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:02.764013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a38a7d-bae2-49fe-abf4-3f1a5ff2812b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:02.804641 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d543e67-a37a-45a6-bdb3-43fdaff17b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:02.805576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d543e67-a37a-45a6-bdb3-43fdaff17b23 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:02.805894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d543e67-a37a-45a6-bdb3-43fdaff17b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:02.806172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d543e67-a37a-45a6-bdb3-43fdaff17b23 None admin] PATH_INFO: `/v3/system/users/429aaa62ee4c4faeae0637a870cb0355/roles/48f409e647e64284817bad2b8ec30bf2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:02.807752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d543e67-a37a-45a6-bdb3-43fdaff17b23 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=429aaa62ee4c4faeae0637a870cb0355, role_id=48f409e647e64284817bad2b8ec30bf2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:02.809173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d543e67-a37a-45a6-bdb3-43fdaff17b23 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:02.819920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 169/334] 10.222.0.43 () {58 vars in 1334 bytes} [Mon Jul 4 07:50:02 2022] PUT /identity/v3/system/users/429aaa62ee4c4faeae0637a870cb0355/roles/48f409e647e64284817bad2b8ec30bf2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:50:04.484531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb57bb-fed2-40b3-bbab-1cdb0ca44fe0 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:04.485644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb57bb-fed2-40b3-bbab-1cdb0ca44fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:04.486146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb57bb-fed2-40b3-bbab-1cdb0ca44fe0 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:04.487892 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 166/335] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:04.494513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9b5b8-09f5-49dc-b59d-32cc8cf07970 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:04.495500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9b5b8-09f5-49dc-b59d-32cc8cf07970 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:04.496100 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9b5b8-09f5-49dc-b59d-32cc8cf07970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:04.522208 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-61c9b5b8-09f5-49dc-b59d-32cc8cf07970 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:04.524293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-61c9b5b8-09f5-49dc-b59d-32cc8cf07970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:04.615486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 170/336] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:04.664881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb57bb-fed2-40b3-bbab-1cdb0ca44fe0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:04.826535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb57bb-fed2-40b3-bbab-1cdb0ca44fe0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:04.857600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:04.874759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1425255-31c6-42ff-9d66-914ad88fe7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:04.875597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1425255-31c6-42ff-9d66-914ad88fe7db None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:04.875884 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1425255-31c6-42ff-9d66-914ad88fe7db None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:04.876165 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1425255-31c6-42ff-9d66-914ad88fe7db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:04.891595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1425255-31c6-42ff-9d66-914ad88fe7db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c1425255-31c6-42ff-9d66-914ad88fe7db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:04.901178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:04.902257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:04.903077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 167/337] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:50:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:04.908845 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9b5b8-09f5-49dc-b59d-32cc8cf07970 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:04.920190 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9ac99-89ae-42f3-9083-c408592b13d0 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:04.920505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9ac99-89ae-42f3-9083-c408592b13d0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:04.920755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9ac99-89ae-42f3-9083-c408592b13d0 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:04.922209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 168/338] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:04.929507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827d3c5-090f-4354-bb18-d65abc3524c4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:04.930527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827d3c5-090f-4354-bb18-d65abc3524c4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:04.930527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827d3c5-090f-4354-bb18-d65abc3524c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:04.967238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f827d3c5-090f-4354-bb18-d65abc3524c4 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:04.969508 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f827d3c5-090f-4354-bb18-d65abc3524c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:05.116144 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 169/339] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:05.130090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:05.150852 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827d3c5-090f-4354-bb18-d65abc3524c4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:05.158613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9b5b8-09f5-49dc-b59d-32cc8cf07970 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:05.248344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272b9e0-fb95-45e2-b673-ebe6b72f34db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:05.249387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272b9e0-fb95-45e2-b673-ebe6b72f34db None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:05.249838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272b9e0-fb95-45e2-b673-ebe6b72f34db None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:05.250263 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272b9e0-fb95-45e2-b673-ebe6b72f34db None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:05.251467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f272b9e0-fb95-45e2-b673-ebe6b72f34db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:05.251934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f272b9e0-fb95-45e2-b673-ebe6b72f34db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:05.254565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f272b9e0-fb95-45e2-b673-ebe6b72f34db None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:05.288630 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 171/340] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:05.293155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272b9e0-fb95-45e2-b673-ebe6b72f34db None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:05.425948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827d3c5-090f-4354-bb18-d65abc3524c4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:05.454004 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:05.510590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6273fc1-7fc6-40bf-b90a-abf3225e3ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:05.511452 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6273fc1-7fc6-40bf-b90a-abf3225e3ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:05.511735 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6273fc1-7fc6-40bf-b90a-abf3225e3ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:05.512018 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6273fc1-7fc6-40bf-b90a-abf3225e3ef4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:05.533082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6273fc1-7fc6-40bf-b90a-abf3225e3ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:05.533729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272b9e0-fb95-45e2-b673-ebe6b72f34db None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d6273fc1-7fc6-40bf-b90a-abf3225e3ef4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:05.546223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:05.547258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:05.548102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 170/341] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:05.549652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e8a4f-fc42-431f-83e6-101d962b4740 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:05.549652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e8a4f-fc42-431f-83e6-101d962b4740 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:05.553975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e8a4f-fc42-431f-83e6-101d962b4740 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:05.555918 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 171/342] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:05.561596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e8a4f-fc42-431f-83e6-101d962b4740 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:05.606208 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9c904-846d-4bbc-9abd-af98cb7c54ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:05.607292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9c904-846d-4bbc-9abd-af98cb7c54ab None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:05.607775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9c904-846d-4bbc-9abd-af98cb7c54ab None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:05.608231 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9c904-846d-4bbc-9abd-af98cb7c54ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:05.655162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:05.773025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9c904-846d-4bbc-9abd-af98cb7c54ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:05.774965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9c904-846d-4bbc-9abd-af98cb7c54ab None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:05.807650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e8a4f-fc42-431f-83e6-101d962b4740 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:05.856676 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71815a36-f6c1-4249-b610-01d971437be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:05.857542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71815a36-f6c1-4249-b610-01d971437be3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:05.857953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71815a36-f6c1-4249-b610-01d971437be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:05.858335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71815a36-f6c1-4249-b610-01d971437be3 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:05.861492 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71815a36-f6c1-4249-b610-01d971437be3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:05.861492 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71815a36-f6c1-4249-b610-01d971437be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:05.861492 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71815a36-f6c1-4249-b610-01d971437be3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:05.873043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 172/343] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:50:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:05.877431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d240c-e3f2-420c-911a-3ec16b953679 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:05.877685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d240c-e3f2-420c-911a-3ec16b953679 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:05.878008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d240c-e3f2-420c-911a-3ec16b953679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:05.900469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-847d240c-e3f2-420c-911a-3ec16b953679 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:05.902148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-847d240c-e3f2-420c-911a-3ec16b953679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:05.949201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:05.952668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 172/344] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:05.954091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9c904-846d-4bbc-9abd-af98cb7c54ab None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:06.025325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:06.035610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 173/345] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:06.037557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d240c-e3f2-420c-911a-3ec16b953679 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:06.131304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9c904-846d-4bbc-9abd-af98cb7c54ab None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:06.184494 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303befb-c178-4f6e-ba26-dcfe37fd0acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:06.186039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303befb-c178-4f6e-ba26-dcfe37fd0acc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:06.186039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303befb-c178-4f6e-ba26-dcfe37fd0acc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:06.186039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303befb-c178-4f6e-ba26-dcfe37fd0acc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:06.210852 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d240c-e3f2-420c-911a-3ec16b953679 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:06.258938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15a50d-d0af-48c3-8509-74bb7c13320e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:06.260155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15a50d-d0af-48c3-8509-74bb7c13320e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:06.260596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15a50d-d0af-48c3-8509-74bb7c13320e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:06.261014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15a50d-d0af-48c3-8509-74bb7c13320e None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:06.262734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15a50d-d0af-48c3-8509-74bb7c13320e None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:06.265436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15a50d-d0af-48c3-8509-74bb7c13320e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:06.267628 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 174/346] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:06 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:06.269308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04158f-0c03-4dfd-81a4-4e09216a9fa8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:06.269833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04158f-0c03-4dfd-81a4-4e09216a9fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:06.270344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04158f-0c03-4dfd-81a4-4e09216a9fa8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:06.271868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 175/347] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:06.273195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04158f-0c03-4dfd-81a4-4e09216a9fa8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:06.283706 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303befb-c178-4f6e-ba26-dcfe37fd0acc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:06.285935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303befb-c178-4f6e-ba26-dcfe37fd0acc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:06.327424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:06.366291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:06.437378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 173/348] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:06.438306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303befb-c178-4f6e-ba26-dcfe37fd0acc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:06.443361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04158f-0c03-4dfd-81a4-4e09216a9fa8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:06.500910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a7af7-a72d-42d0-a234-bb0b3318e93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:06.502535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a7af7-a72d-42d0-a234-bb0b3318e93e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:06.502819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a7af7-a72d-42d0-a234-bb0b3318e93e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:06.503094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a7af7-a72d-42d0-a234-bb0b3318e93e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:06.516698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a7af7-a72d-42d0-a234-bb0b3318e93e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b72a7af7-a72d-42d0-a234-bb0b3318e93e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:06.524767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:06.526220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:06.527063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 176/349] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:06.527960 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3cb49-2939-4b27-a1f7-98c1be0341fa None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:06.528090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3cb49-2939-4b27-a1f7-98c1be0341fa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:06.528369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3cb49-2939-4b27-a1f7-98c1be0341fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:06.550711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-46a3cb49-2939-4b27-a1f7-98c1be0341fa None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:06.551851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-46a3cb49-2939-4b27-a1f7-98c1be0341fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:06.567468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303befb-c178-4f6e-ba26-dcfe37fd0acc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:06.620919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a68dbb-2221-4221-9288-3844026822bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:06.621695 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a68dbb-2221-4221-9288-3844026822bf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:06.621987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a68dbb-2221-4221-9288-3844026822bf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:06.622267 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a68dbb-2221-4221-9288-3844026822bf None admin] PATH_INFO: `/v3/groups/84dfd81bd5b04b0580f570753687e1ec` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:06.631788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a68dbb-2221-4221-9288-3844026822bf None admin] RBAC: Authorizing `identity:get_group(group_id=84dfd81bd5b04b0580f570753687e1ec)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:06.634172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a68dbb-2221-4221-9288-3844026822bf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:06.637395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 174/350] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:06 2022] GET /identity/v3/groups/84dfd81bd5b04b0580f570753687e1ec => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:06.639514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc4eeb-cad8-4ea8-be1d-00868a728536 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:06.640071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc4eeb-cad8-4ea8-be1d-00868a728536 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:06.641181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc4eeb-cad8-4ea8-be1d-00868a728536 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:06.643321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 175/351] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:06.645351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc4eeb-cad8-4ea8-be1d-00868a728536 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:06.654828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 177/352] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:06.656454 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3cb49-2939-4b27-a1f7-98c1be0341fa None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:06.669034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:06.691598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:06.795210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3cb49-2939-4b27-a1f7-98c1be0341fa None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:06.795586 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc4eeb-cad8-4ea8-be1d-00868a728536 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:06.860921 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45853b2-5592-411c-8994-ee7a001247ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:06.861524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45853b2-5592-411c-8994-ee7a001247ed None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:06.862531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45853b2-5592-411c-8994-ee7a001247ed None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:06.863150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45853b2-5592-411c-8994-ee7a001247ed None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:06.865173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45853b2-5592-411c-8994-ee7a001247ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:06.865173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45853b2-5592-411c-8994-ee7a001247ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:06.870136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45853b2-5592-411c-8994-ee7a001247ed None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:06.881084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f7b6e-50c3-472b-bab6-eeaccd46b8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:06.882756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f7b6e-50c3-472b-bab6-eeaccd46b8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:06.883267 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f7b6e-50c3-472b-bab6-eeaccd46b8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:06.884382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f7b6e-50c3-472b-bab6-eeaccd46b8c5 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:06.886582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 178/353] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:50:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:06.886705 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f7b6e-50c3-472b-bab6-eeaccd46b8c5 None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:06.887778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45853b2-5592-411c-8994-ee7a001247ed None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:06.892952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f7b6e-50c3-472b-bab6-eeaccd46b8c5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:06.893162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 176/354] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:06 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:06.897322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942ec93-1403-4a83-9862-45672fecb453 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:06.897910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942ec93-1403-4a83-9862-45672fecb453 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:06.899335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942ec93-1403-4a83-9862-45672fecb453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:06.933306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e942ec93-1403-4a83-9862-45672fecb453 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:06.935466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e942ec93-1403-4a83-9862-45672fecb453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:07.016252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:07.042706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45853b2-5592-411c-8994-ee7a001247ed None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:07.071913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 177/355] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:07.072797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942ec93-1403-4a83-9862-45672fecb453 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:07.088391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:07.129587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3708008-badf-4be9-a40d-6606cc6d21b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:07.130255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3708008-badf-4be9-a40d-6606cc6d21b0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:07.130602 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3708008-badf-4be9-a40d-6606cc6d21b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:07.131000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3708008-badf-4be9-a40d-6606cc6d21b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:07.132040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3708008-badf-4be9-a40d-6606cc6d21b0 None admin] RBAC: Adding query filter params (group.id=84dfd81bd5b04b0580f570753687e1ec, role.id=794ff5f5032d40a98e67fb0df718b75c, scope.project.id=d44ad181a25048dea48323087ad2f3ca) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:07.132586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3708008-badf-4be9-a40d-6606cc6d21b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:07.135340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3708008-badf-4be9-a40d-6606cc6d21b0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:07.143942 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 179/356] 10.222.0.43 () {56 vars in 1376 bytes} [Mon Jul 4 07:50:06 2022] GET /identity/v3/role_assignments?group.id=84dfd81bd5b04b0580f570753687e1ec&scope.project.id=d44ad181a25048dea48323087ad2f3ca&role.id=794ff5f5032d40a98e67fb0df718b75c => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:07.144827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3708008-badf-4be9-a40d-6606cc6d21b0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:07.218842 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942ec93-1403-4a83-9862-45672fecb453 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:07.292454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fdba8-62a0-46db-a2ac-8367fd39990a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:07.293781 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fdba8-62a0-46db-a2ac-8367fd39990a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:07.294446 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fdba8-62a0-46db-a2ac-8367fd39990a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:07.294934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fdba8-62a0-46db-a2ac-8367fd39990a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:07.316831 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fdba8-62a0-46db-a2ac-8367fd39990a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:07.329288 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3708008-badf-4be9-a40d-6606cc6d21b0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ba1fdba8-62a0-46db-a2ac-8367fd39990a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:07.331521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:07.332621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:07.333917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 178/357] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:50:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:07.336338 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fdba8-62a0-46db-a2ac-8367fd39990a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:07.399993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236f729-ddff-4a65-87eb-d8988e471e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:07.400831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f729-ddff-4a65-87eb-d8988e471e01 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:07.401114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f729-ddff-4a65-87eb-d8988e471e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:07.401392 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f729-ddff-4a65-87eb-d8988e471e01 None admin] PATH_INFO: `/v3/groups/84dfd81bd5b04b0580f570753687e1ec` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:07.402812 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236f729-ddff-4a65-87eb-d8988e471e01 None admin] RBAC: Authorizing `identity:get_group(group_id=84dfd81bd5b04b0580f570753687e1ec)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:07.404353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236f729-ddff-4a65-87eb-d8988e471e01 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:07.406385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 180/358] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:07 2022] GET /identity/v3/groups/84dfd81bd5b04b0580f570753687e1ec => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:07.407048 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236f729-ddff-4a65-87eb-d8988e471e01 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:07.438715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:07.478044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:07.533375 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fdba8-62a0-46db-a2ac-8367fd39990a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:07.592943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792e4fb-6660-4304-9e54-add84253dde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:07.594131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792e4fb-6660-4304-9e54-add84253dde9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:07.594565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792e4fb-6660-4304-9e54-add84253dde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:07.594981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792e4fb-6660-4304-9e54-add84253dde9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:07.619259 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236f729-ddff-4a65-87eb-d8988e471e01 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:07.687173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7338c5-ca81-44af-beae-1a549c22bbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:07.688986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7338c5-ca81-44af-beae-1a549c22bbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:07.689482 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7338c5-ca81-44af-beae-1a549c22bbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:07.690069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7338c5-ca81-44af-beae-1a549c22bbf5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:07.712079 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7338c5-ca81-44af-beae-1a549c22bbf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:07.725276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792e4fb-6660-4304-9e54-add84253dde9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9b7338c5-ca81-44af-beae-1a549c22bbf5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:07.727775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:07.728821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:07.729848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:07.729848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:07.729848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:07.729848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:07.729848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:07.729848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:07.729848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:07.729848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:07.729848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:07.729848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:07.729848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 07:50:07.729848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:07.730475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792e4fb-6660-4304-9e54-add84253dde9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:07.732278 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 181/359] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:07.732911 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830572fc-bc2b-419d-863b-39fb0cbba748 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:07.733409 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830572fc-bc2b-419d-863b-39fb0cbba748 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:07.733950 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830572fc-bc2b-419d-863b-39fb0cbba748 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:07.737103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 182/360] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:07.738724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830572fc-bc2b-419d-863b-39fb0cbba748 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:07.886976 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:07.903599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:07.933010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 179/361] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:07.934166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792e4fb-6660-4304-9e54-add84253dde9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:07.944213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830572fc-bc2b-419d-863b-39fb0cbba748 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:07.990294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffd8f7-192d-46d0-abad-ab48d94310ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:07.991278 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffd8f7-192d-46d0-abad-ab48d94310ac None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:07.991737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffd8f7-192d-46d0-abad-ab48d94310ac None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:07.992556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffd8f7-192d-46d0-abad-ab48d94310ac None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:07.993515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffd8f7-192d-46d0-abad-ab48d94310ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:07.994008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffd8f7-192d-46d0-abad-ab48d94310ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:07.995630 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffd8f7-192d-46d0-abad-ab48d94310ac None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:08.007372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 183/362] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:08.008365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffd8f7-192d-46d0-abad-ab48d94310ac None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:08.092757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792e4fb-6660-4304-9e54-add84253dde9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:08.169026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d16d12-7b20-4e60-9707-46e8932b9d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:08.170224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d16d12-7b20-4e60-9707-46e8932b9d36 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:08.170725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d16d12-7b20-4e60-9707-46e8932b9d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:08.171207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d16d12-7b20-4e60-9707-46e8932b9d36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:08.172308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d16d12-7b20-4e60-9707-46e8932b9d36 None admin] RBAC: Adding query filter params (group.id=84dfd81bd5b04b0580f570753687e1ec, role.id=3edce31396af478ea1ea7b76d33da530, scope.project.id=d44ad181a25048dea48323087ad2f3ca) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:08.172856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d16d12-7b20-4e60-9707-46e8932b9d36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:08.174659 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d16d12-7b20-4e60-9707-46e8932b9d36 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:08.182773 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 180/363] 10.222.0.43 () {56 vars in 1376 bytes} [Mon Jul 4 07:50:07 2022] GET /identity/v3/role_assignments?group.id=84dfd81bd5b04b0580f570753687e1ec&scope.project.id=d44ad181a25048dea48323087ad2f3ca&role.id=3edce31396af478ea1ea7b76d33da530 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:08.184592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48fa61-0954-4fc7-bfb1-d9db67a1d90b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:08.185120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48fa61-0954-4fc7-bfb1-d9db67a1d90b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:08.185603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48fa61-0954-4fc7-bfb1-d9db67a1d90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:08.212629 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4e48fa61-0954-4fc7-bfb1-d9db67a1d90b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:08.214390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4e48fa61-0954-4fc7-bfb1-d9db67a1d90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:08.214531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffd8f7-192d-46d0-abad-ab48d94310ac None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:08.291341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0236072-3233-47d3-9de5-a26795276574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:08.292468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:08.293322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0236072-3233-47d3-9de5-a26795276574 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:08.294064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0236072-3233-47d3-9de5-a26795276574 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:08.294853 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0236072-3233-47d3-9de5-a26795276574 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:08.296539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0236072-3233-47d3-9de5-a26795276574 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:08.297356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0236072-3233-47d3-9de5-a26795276574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:08.300513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0236072-3233-47d3-9de5-a26795276574 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:08.312774 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:08.329443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 184/364] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:50:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:08.331513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0236072-3233-47d3-9de5-a26795276574 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:08.383883 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 181/365] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:08.385418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48fa61-0954-4fc7-bfb1-d9db67a1d90b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:08.573192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0236072-3233-47d3-9de5-a26795276574 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:08.622946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48fa61-0954-4fc7-bfb1-d9db67a1d90b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:08.652652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac1d61-df3a-4b48-b14c-5a95227efb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:08.655260 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac1d61-df3a-4b48-b14c-5a95227efb48 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:08.656034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac1d61-df3a-4b48-b14c-5a95227efb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:08.656776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac1d61-df3a-4b48-b14c-5a95227efb48 None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:08.659322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac1d61-df3a-4b48-b14c-5a95227efb48 None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:08.662287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac1d61-df3a-4b48-b14c-5a95227efb48 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:08.666451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 185/366] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:08 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:08.668231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac1d61-df3a-4b48-b14c-5a95227efb48 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:08.716229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff533c4-75ec-4dc4-86e9-ac052846d65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:08.718672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff533c4-75ec-4dc4-86e9-ac052846d65b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:08.719230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff533c4-75ec-4dc4-86e9-ac052846d65b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:08.719741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff533c4-75ec-4dc4-86e9-ac052846d65b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:08.801775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:08.830561 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:08.863443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff533c4-75ec-4dc4-86e9-ac052846d65b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:08.865306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff533c4-75ec-4dc4-86e9-ac052846d65b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:08.912534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac1d61-df3a-4b48-b14c-5a95227efb48 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:08.970714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697dd02-14d7-42ae-9600-318b89f14532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:08.971624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697dd02-14d7-42ae-9600-318b89f14532 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:08.971908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697dd02-14d7-42ae-9600-318b89f14532 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:08.972173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697dd02-14d7-42ae-9600-318b89f14532 None admin] PATH_INFO: `/v3/users/429aaa62ee4c4faeae0637a870cb0355` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:08.974410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697dd02-14d7-42ae-9600-318b89f14532 None admin] RBAC: Authorizing `identity:get_user(user_id=429aaa62ee4c4faeae0637a870cb0355)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:08.975899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697dd02-14d7-42ae-9600-318b89f14532 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:08.978169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 186/367] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:50:08 2022] GET /identity/v3/users/429aaa62ee4c4faeae0637a870cb0355 => generated 293 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:08.978986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697dd02-14d7-42ae-9600-318b89f14532 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:09.094862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 182/368] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:09.096783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff533c4-75ec-4dc4-86e9-ac052846d65b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:09.102198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697dd02-14d7-42ae-9600-318b89f14532 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:09.145244 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebe548-0733-4f06-be8d-57184d7b50f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:09.146092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebe548-0733-4f06-be8d-57184d7b50f9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:09.146370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebe548-0733-4f06-be8d-57184d7b50f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:09.146696 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebe548-0733-4f06-be8d-57184d7b50f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:09.149893 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:09.159801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebe548-0733-4f06-be8d-57184d7b50f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f1ebe548-0733-4f06-be8d-57184d7b50f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:09.168114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:09.169790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:09.171210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:09.171210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:09.171210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:09.171210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:09.171210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:09.171210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:09.171210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:09.171210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:09.171210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:09.171210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:09.171210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:50:09.171210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:09.177664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 187/369] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:50:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:09.177664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebe548-0733-4f06-be8d-57184d7b50f9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:09.230576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:09.289263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebe548-0733-4f06-be8d-57184d7b50f9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:09.303819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff533c4-75ec-4dc4-86e9-ac052846d65b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:09.328800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03af4-287e-480b-8845-7e65c8e5fce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:09.329473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03af4-287e-480b-8845-7e65c8e5fce8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:09.329793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03af4-287e-480b-8845-7e65c8e5fce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:09.330028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03af4-287e-480b-8845-7e65c8e5fce8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:09.330893 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da03af4-287e-480b-8845-7e65c8e5fce8 None admin] RBAC: Adding query filter params (role.id=48f409e647e64284817bad2b8ec30bf2, user.id=429aaa62ee4c4faeae0637a870cb0355, scope.system=all) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:09.331202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da03af4-287e-480b-8845-7e65c8e5fce8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:09.334085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da03af4-287e-480b-8845-7e65c8e5fce8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:09.348321 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 188/370] 10.222.0.43 () {56 vars in 1308 bytes} [Mon Jul 4 07:50:09 2022] GET /identity/v3/role_assignments?user.id=429aaa62ee4c4faeae0637a870cb0355&scope.system=all&role.id=48f409e647e64284817bad2b8ec30bf2 => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:09.349104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03af4-287e-480b-8845-7e65c8e5fce8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:09.376514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570c6f9-d250-479d-8192-f1548502f2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:09.378126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570c6f9-d250-479d-8192-f1548502f2ef None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:09.378630 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570c6f9-d250-479d-8192-f1548502f2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:09.379429 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570c6f9-d250-479d-8192-f1548502f2ef None admin] PATH_INFO: `/v3/groups/84dfd81bd5b04b0580f570753687e1ec` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:09.381893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570c6f9-d250-479d-8192-f1548502f2ef None admin] RBAC: Authorizing `identity:get_group(group_id=84dfd81bd5b04b0580f570753687e1ec)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:09.385412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570c6f9-d250-479d-8192-f1548502f2ef None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:09.388718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 183/371] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:09 2022] GET /identity/v3/groups/84dfd81bd5b04b0580f570753687e1ec => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:09.390200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570c6f9-d250-479d-8192-f1548502f2ef None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:09.462580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:09.582521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03af4-287e-480b-8845-7e65c8e5fce8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:09.621679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d152e-e1d3-4110-90a6-32349eb791d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:09.622485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d152e-e1d3-4110-90a6-32349eb791d8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:09.622761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d152e-e1d3-4110-90a6-32349eb791d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:09.623033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d152e-e1d3-4110-90a6-32349eb791d8 None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:09.624806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d152e-e1d3-4110-90a6-32349eb791d8 None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:09.626675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d152e-e1d3-4110-90a6-32349eb791d8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:09.628452 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 189/372] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:09 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:09.637979 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d152e-e1d3-4110-90a6-32349eb791d8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:09.689966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570c6f9-d250-479d-8192-f1548502f2ef None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:09.748952 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d152e-e1d3-4110-90a6-32349eb791d8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:09.766015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6678f76-85ac-44e2-8b64-dd313fff93a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:09.767616 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6678f76-85ac-44e2-8b64-dd313fff93a5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:09.768375 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6678f76-85ac-44e2-8b64-dd313fff93a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:09.769280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6678f76-85ac-44e2-8b64-dd313fff93a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:09.770933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6678f76-85ac-44e2-8b64-dd313fff93a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:09.771754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6678f76-85ac-44e2-8b64-dd313fff93a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:09.779313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:09.781406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6678f76-85ac-44e2-8b64-dd313fff93a5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:09.784632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:09.794019 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3038ec-48bd-45d5-9ba9-751a1312cde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:09.794696 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3038ec-48bd-45d5-9ba9-751a1312cde3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:09.795002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3038ec-48bd-45d5-9ba9-751a1312cde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:09.795237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3038ec-48bd-45d5-9ba9-751a1312cde3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:09.796088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3038ec-48bd-45d5-9ba9-751a1312cde3 None admin] RBAC: Adding query filter params (group.id=84dfd81bd5b04b0580f570753687e1ec, role.id=3edce31396af478ea1ea7b76d33da530, scope.project.id=e443cfb7bf354891a724342b87c62cda) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:09.796392 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3038ec-48bd-45d5-9ba9-751a1312cde3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:09.798084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3038ec-48bd-45d5-9ba9-751a1312cde3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:09.804891 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 190/373] 10.222.0.43 () {56 vars in 1376 bytes} [Mon Jul 4 07:50:09 2022] GET /identity/v3/role_assignments?group.id=84dfd81bd5b04b0580f570753687e1ec&scope.project.id=e443cfb7bf354891a724342b87c62cda&role.id=3edce31396af478ea1ea7b76d33da530 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:09.805774 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3038ec-48bd-45d5-9ba9-751a1312cde3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:09.807811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 184/374] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:09.818624 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6678f76-85ac-44e2-8b64-dd313fff93a5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:10.001357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3038ec-48bd-45d5-9ba9-751a1312cde3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:10.014914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6678f76-85ac-44e2-8b64-dd313fff93a5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:10.073669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab461cff-6dd5-42c7-9d4f-6651f7fd4cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:10.073669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab461cff-6dd5-42c7-9d4f-6651f7fd4cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:10.073669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab461cff-6dd5-42c7-9d4f-6651f7fd4cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:10.073669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab461cff-6dd5-42c7-9d4f-6651f7fd4cf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:10.091063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0e621-9002-42ce-8df5-f1508555567a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:10.091985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0e621-9002-42ce-8df5-f1508555567a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:10.092269 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0e621-9002-42ce-8df5-f1508555567a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:10.092541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0e621-9002-42ce-8df5-f1508555567a None admin] PATH_INFO: `/v3/groups/84dfd81bd5b04b0580f570753687e1ec` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:10.097292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0e621-9002-42ce-8df5-f1508555567a None admin] RBAC: Authorizing `identity:get_group(group_id=84dfd81bd5b04b0580f570753687e1ec)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:10.099871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0e621-9002-42ce-8df5-f1508555567a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:10.107025 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 191/375] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:09 2022] GET /identity/v3/groups/84dfd81bd5b04b0580f570753687e1ec => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:10.112304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0e621-9002-42ce-8df5-f1508555567a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:10.206309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:10.265262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0e621-9002-42ce-8df5-f1508555567a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:10.293889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:10.310808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86909449-01e3-4c85-8c21-524a67a7c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:10.311506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86909449-01e3-4c85-8c21-524a67a7c1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:10.311923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86909449-01e3-4c85-8c21-524a67a7c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:10.312212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86909449-01e3-4c85-8c21-524a67a7c1a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:10.313116 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86909449-01e3-4c85-8c21-524a67a7c1a6 None admin] RBAC: Adding query filter params (group.id=84dfd81bd5b04b0580f570753687e1ec, role.id=794ff5f5032d40a98e67fb0df718b75c, scope.project.id=e443cfb7bf354891a724342b87c62cda) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:10.313428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86909449-01e3-4c85-8c21-524a67a7c1a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:10.315027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86909449-01e3-4c85-8c21-524a67a7c1a6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:10.322192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 192/376] 10.222.0.43 () {56 vars in 1376 bytes} [Mon Jul 4 07:50:10 2022] GET /identity/v3/role_assignments?group.id=84dfd81bd5b04b0580f570753687e1ec&scope.project.id=e443cfb7bf354891a724342b87c62cda&role.id=794ff5f5032d40a98e67fb0df718b75c => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:10.323900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab461cff-6dd5-42c7-9d4f-6651f7fd4cf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:10.326734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab461cff-6dd5-42c7-9d4f-6651f7fd4cf8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:10.538739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 185/377] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 721 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 07:50:10.545655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390cdc8-df59-4e8e-bd26-2434f7aa2093 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:10.546138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390cdc8-df59-4e8e-bd26-2434f7aa2093 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:10.546596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390cdc8-df59-4e8e-bd26-2434f7aa2093 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:10.547229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab461cff-6dd5-42c7-9d4f-6651f7fd4cf8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:10.553838 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 193/378] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:50:10.557965 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082608c2-1177-4a6c-a6f0-6e1a19afb0e9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:10.558503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082608c2-1177-4a6c-a6f0-6e1a19afb0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:10.559012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082608c2-1177-4a6c-a6f0-6e1a19afb0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:10.594791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-082608c2-1177-4a6c-a6f0-6e1a19afb0e9 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:10.597642 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-082608c2-1177-4a6c-a6f0-6e1a19afb0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:10.749800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 194/379] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:10.771707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:10.781596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab461cff-6dd5-42c7-9d4f-6651f7fd4cf8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:10.795776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082608c2-1177-4a6c-a6f0-6e1a19afb0e9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:10.878222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0ca9d-dff1-4744-a4dc-df966b8a70a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:10.880175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0ca9d-dff1-4744-a4dc-df966b8a70a6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:10.880639 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0ca9d-dff1-4744-a4dc-df966b8a70a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:10.881168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0ca9d-dff1-4744-a4dc-df966b8a70a6 None admin] PATH_INFO: `/v3/projects/cf3a0790796c49f294035007b8a8c25a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:10.884250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0ca9d-dff1-4744-a4dc-df966b8a70a6 None admin] RBAC: Authorizing `identity:get_project(project_id=cf3a0790796c49f294035007b8a8c25a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:10.889918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0ca9d-dff1-4744-a4dc-df966b8a70a6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:10.893744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 186/380] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:10 2022] GET /identity/v3/projects/cf3a0790796c49f294035007b8a8c25a => generated 344 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:10.912328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0ca9d-dff1-4744-a4dc-df966b8a70a6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:10.917262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:11.070860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082608c2-1177-4a6c-a6f0-6e1a19afb0e9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:11.084412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0ca9d-dff1-4744-a4dc-df966b8a70a6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:11.147040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c283a6a-f908-427f-838e-e3beb70e7a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:11.148534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c283a6a-f908-427f-838e-e3beb70e7a64 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:11.149019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c283a6a-f908-427f-838e-e3beb70e7a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:11.149491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c283a6a-f908-427f-838e-e3beb70e7a64 None admin] PATH_INFO: `/v3/groups/43db6823e7dd4683a5f5a2c2959214f6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:11.157686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeb154-f24c-473b-bb12-0b7ccdfd96c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:11.158546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeb154-f24c-473b-bb12-0b7ccdfd96c5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:11.158828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeb154-f24c-473b-bb12-0b7ccdfd96c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:11.159103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeb154-f24c-473b-bb12-0b7ccdfd96c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:11.163148 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c283a6a-f908-427f-838e-e3beb70e7a64 None admin] RBAC: Authorizing `identity:get_group(group_id=43db6823e7dd4683a5f5a2c2959214f6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:11.165566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c283a6a-f908-427f-838e-e3beb70e7a64 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:11.170908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 187/381] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:10 2022] GET /identity/v3/groups/43db6823e7dd4683a5f5a2c2959214f6 => generated 229 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:11.171191 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eeb154-f24c-473b-bb12-0b7ccdfd96c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:11.178416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c283a6a-f908-427f-838e-e3beb70e7a64 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c7eeb154-f24c-473b-bb12-0b7ccdfd96c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:11.179648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:11.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:11.182742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 195/382] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:50:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:11.189032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeb154-f24c-473b-bb12-0b7ccdfd96c5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:11.278904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:11.347412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeb154-f24c-473b-bb12-0b7ccdfd96c5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:11.365769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:11.437708 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caff648-9bcd-4e0d-87fe-7d2f716b1203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:11.438752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caff648-9bcd-4e0d-87fe-7d2f716b1203 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:11.439260 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caff648-9bcd-4e0d-87fe-7d2f716b1203 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:11.439676 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caff648-9bcd-4e0d-87fe-7d2f716b1203 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:11.440967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caff648-9bcd-4e0d-87fe-7d2f716b1203 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:11.441437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caff648-9bcd-4e0d-87fe-7d2f716b1203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:11.443818 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caff648-9bcd-4e0d-87fe-7d2f716b1203 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:11.458961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 196/383] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:11.467284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caff648-9bcd-4e0d-87fe-7d2f716b1203 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:11.489899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c283a6a-f908-427f-838e-e3beb70e7a64 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:11.563849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1330a-8789-497c-8053-970884e5b1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:11.564543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1330a-8789-497c-8053-970884e5b1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:11.564820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1330a-8789-497c-8053-970884e5b1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:11.565093 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1330a-8789-497c-8053-970884e5b1d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:11.565996 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1330a-8789-497c-8053-970884e5b1d7 None admin] RBAC: Adding query filter params (group.id=43db6823e7dd4683a5f5a2c2959214f6, role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=cf3a0790796c49f294035007b8a8c25a) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:11.566307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1330a-8789-497c-8053-970884e5b1d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:11.567790 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1330a-8789-497c-8053-970884e5b1d7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:11.580800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 188/384] 10.222.0.43 () {56 vars in 1376 bytes} [Mon Jul 4 07:50:11 2022] GET /identity/v3/role_assignments?group.id=43db6823e7dd4683a5f5a2c2959214f6&scope.project.id=cf3a0790796c49f294035007b8a8c25a&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 260 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:11.693565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caff648-9bcd-4e0d-87fe-7d2f716b1203 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:11.745467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d49c4-80cd-4f0b-a811-52d11362f123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:11.746497 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d49c4-80cd-4f0b-a811-52d11362f123 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:11.746774 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d49c4-80cd-4f0b-a811-52d11362f123 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:11.747046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d49c4-80cd-4f0b-a811-52d11362f123 None admin] PATH_INFO: `/v3/groups/84dfd81bd5b04b0580f570753687e1ec` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:11.748413 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d49c4-80cd-4f0b-a811-52d11362f123 None admin] RBAC: Authorizing `identity:get_group(group_id=84dfd81bd5b04b0580f570753687e1ec)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:11.749993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d49c4-80cd-4f0b-a811-52d11362f123 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:11.752107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 197/385] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:11 2022] GET /identity/v3/groups/84dfd81bd5b04b0580f570753687e1ec => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:11.758854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1330a-8789-497c-8053-970884e5b1d7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:11.906335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:11.916557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1330a-8789-497c-8053-970884e5b1d7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:11.955634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f7dec-c00a-4c48-aeac-b3b4dba92751 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:11.956238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f7dec-c00a-4c48-aeac-b3b4dba92751 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:11.956749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f7dec-c00a-4c48-aeac-b3b4dba92751 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:11.958762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 198/386] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:11.974598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a495bb1-9f6a-44e1-856d-3988efd2cbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:11.976173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a495bb1-9f6a-44e1-856d-3988efd2cbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:11.976683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a495bb1-9f6a-44e1-856d-3988efd2cbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:11.977176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a495bb1-9f6a-44e1-856d-3988efd2cbe2 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:11.982796 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a495bb1-9f6a-44e1-856d-3988efd2cbe2 None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:11.983912 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339941bd-8fd6-46e0-bdc9-f3db8e49a951 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:11.984380 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339941bd-8fd6-46e0-bdc9-f3db8e49a951 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:11.984841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339941bd-8fd6-46e0-bdc9-f3db8e49a951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:11.985516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a495bb1-9f6a-44e1-856d-3988efd2cbe2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:11.989324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 189/387] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:11 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:11.995957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a495bb1-9f6a-44e1-856d-3988efd2cbe2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:12.047405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-339941bd-8fd6-46e0-bdc9-f3db8e49a951 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:12.049470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-339941bd-8fd6-46e0-bdc9-f3db8e49a951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:12.154047 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:12.235378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 199/388] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:12.270143 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339941bd-8fd6-46e0-bdc9-f3db8e49a951 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:12.299960 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a495bb1-9f6a-44e1-856d-3988efd2cbe2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:12.347037 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f36dca-99b8-48fc-a3c4-23f4e14eacfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:12.348531 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f36dca-99b8-48fc-a3c4-23f4e14eacfb None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:12.349014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f36dca-99b8-48fc-a3c4-23f4e14eacfb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:12.349481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f36dca-99b8-48fc-a3c4-23f4e14eacfb None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca/groups/84dfd81bd5b04b0580f570753687e1ec/roles/794ff5f5032d40a98e67fb0df718b75c` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:12.352084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f36dca-99b8-48fc-a3c4-23f4e14eacfb None admin] RBAC: Authorizing `identity:create_grant(project_id=d44ad181a25048dea48323087ad2f3ca, group_id=84dfd81bd5b04b0580f570753687e1ec, role_id=794ff5f5032d40a98e67fb0df718b75c)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:12.353967 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f36dca-99b8-48fc-a3c4-23f4e14eacfb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:12.374583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 190/389] 10.222.0.43 () {58 vars in 1442 bytes} [Mon Jul 4 07:50:11 2022] PUT /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca/groups/84dfd81bd5b04b0580f570753687e1ec/roles/794ff5f5032d40a98e67fb0df718b75c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:50:12.509679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339941bd-8fd6-46e0-bdc9-f3db8e49a951 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:12.567661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf411ac-272e-457d-8139-cba8088c4c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:12.568505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf411ac-272e-457d-8139-cba8088c4c4d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:12.568789 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf411ac-272e-457d-8139-cba8088c4c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:12.569062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf411ac-272e-457d-8139-cba8088c4c4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:12.590986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf411ac-272e-457d-8139-cba8088c4c4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4bf411ac-272e-457d-8139-cba8088c4c4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:12.600700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:12.602417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:12.604298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 200/390] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:12.609505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f36dca-99b8-48fc-a3c4-23f4e14eacfb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:12.662995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:12.776098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f36dca-99b8-48fc-a3c4-23f4e14eacfb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:12.842951 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7bbc4-8ca8-437d-a80b-0ebfb389b9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:12.844560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7bbc4-8ca8-437d-a80b-0ebfb389b9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:12.845347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7bbc4-8ca8-437d-a80b-0ebfb389b9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:12.846138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7bbc4-8ca8-437d-a80b-0ebfb389b9c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:12.847815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7bbc4-8ca8-437d-a80b-0ebfb389b9c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:12.848641 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7bbc4-8ca8-437d-a80b-0ebfb389b9c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:12.850757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7bbc4-8ca8-437d-a80b-0ebfb389b9c1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:12.867618 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 191/391] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:50:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:12.881309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf411ac-272e-457d-8139-cba8088c4c4d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:12.945073 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:13.075269 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf411ac-272e-457d-8139-cba8088c4c4d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:13.143958 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e51d5-fc83-4c36-8fb0-32e14e418dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:13.144863 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e51d5-fc83-4c36-8fb0-32e14e418dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:13.145148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e51d5-fc83-4c36-8fb0-32e14e418dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:13.145422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e51d5-fc83-4c36-8fb0-32e14e418dd0 None admin] PATH_INFO: `/v3/groups/84dfd81bd5b04b0580f570753687e1ec` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:13.146780 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e51d5-fc83-4c36-8fb0-32e14e418dd0 None admin] RBAC: Authorizing `identity:get_group(group_id=84dfd81bd5b04b0580f570753687e1ec)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:13.148354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e51d5-fc83-4c36-8fb0-32e14e418dd0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:13.150620 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 201/392] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:12 2022] GET /identity/v3/groups/84dfd81bd5b04b0580f570753687e1ec => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:13.158816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7bbc4-8ca8-437d-a80b-0ebfb389b9c1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:13.254784 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:13.274926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7bbc4-8ca8-437d-a80b-0ebfb389b9c1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:13.324674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147a157-cf39-4284-ac05-a3f0589e28fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:13.325636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147a157-cf39-4284-ac05-a3f0589e28fa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:13.325872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147a157-cf39-4284-ac05-a3f0589e28fa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:13.326151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147a157-cf39-4284-ac05-a3f0589e28fa None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:13.327352 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147a157-cf39-4284-ac05-a3f0589e28fa None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:13.328815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147a157-cf39-4284-ac05-a3f0589e28fa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:13.336501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e51d5-fc83-4c36-8fb0-32e14e418dd0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:13.337302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 192/393] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:13 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:13.347300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28249c-b777-4d0d-8e4e-daf046789d61 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:13.347931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28249c-b777-4d0d-8e4e-daf046789d61 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:13.348411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28249c-b777-4d0d-8e4e-daf046789d61 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:13.350406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 193/394] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:13.357514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c0130-f20f-42a1-8147-3a9978669c2e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:13.358195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c0130-f20f-42a1-8147-3a9978669c2e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:13.358693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c0130-f20f-42a1-8147-3a9978669c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:13.392864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a61c0130-f20f-42a1-8147-3a9978669c2e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:13.394518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a61c0130-f20f-42a1-8147-3a9978669c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:13.471752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:13.481611 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e51d5-fc83-4c36-8fb0-32e14e418dd0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:13.557519 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 194/395] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:50:13.564181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee69129-0e40-48d1-babe-11b1cb2f44a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:13.565101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee69129-0e40-48d1-babe-11b1cb2f44a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:13.565385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee69129-0e40-48d1-babe-11b1cb2f44a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:13.565678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee69129-0e40-48d1-babe-11b1cb2f44a8 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca/groups/84dfd81bd5b04b0580f570753687e1ec/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:13.570988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee69129-0e40-48d1-babe-11b1cb2f44a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d44ad181a25048dea48323087ad2f3ca, group_id=84dfd81bd5b04b0580f570753687e1ec, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:13.572883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee69129-0e40-48d1-babe-11b1cb2f44a8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:13.605047 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 202/396] 10.222.0.43 () {58 vars in 1442 bytes} [Mon Jul 4 07:50:13 2022] PUT /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca/groups/84dfd81bd5b04b0580f570753687e1ec/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:50:13.610615 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c0130-f20f-42a1-8147-3a9978669c2e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:13.619845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1053a55-84ec-4d56-b5e4-89d191afc3c3 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:13.620192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1053a55-84ec-4d56-b5e4-89d191afc3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:13.620451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1053a55-84ec-4d56-b5e4-89d191afc3c3 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:13.623553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 203/397] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:50:13.630727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc38736-e82b-4ed1-b34c-03dae8012a0c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:13.630727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc38736-e82b-4ed1-b34c-03dae8012a0c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:13.630727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc38736-e82b-4ed1-b34c-03dae8012a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:13.667518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2cc38736-e82b-4ed1-b34c-03dae8012a0c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:13.669405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2cc38736-e82b-4ed1-b34c-03dae8012a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:13.778326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 204/398] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:13.818237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:13.824348 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc38736-e82b-4ed1-b34c-03dae8012a0c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:13.842676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c0130-f20f-42a1-8147-3a9978669c2e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:13.896309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299b204-c145-4cb9-9591-38e5fc5d9b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:13.897057 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299b204-c145-4cb9-9591-38e5fc5d9b60 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:13.897334 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299b204-c145-4cb9-9591-38e5fc5d9b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:13.897606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299b204-c145-4cb9-9591-38e5fc5d9b60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:13.911109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299b204-c145-4cb9-9591-38e5fc5d9b60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b299b204-c145-4cb9-9591-38e5fc5d9b60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:13.919425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:13.921130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:13.923081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:13.923081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:13.923081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:13.923081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:13.923081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:13.923081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:13.923081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:13.923081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:13.923081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:13.923081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:13.923081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:13.923081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:13.925779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 195/399] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:13.927495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299b204-c145-4cb9-9591-38e5fc5d9b60 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:14.011099 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:14.053739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc38736-e82b-4ed1-b34c-03dae8012a0c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:14.053985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299b204-c145-4cb9-9591-38e5fc5d9b60 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:14.104249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6154d1-36a6-48fa-98ed-6a6e648fbd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:14.104950 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6154d1-36a6-48fa-98ed-6a6e648fbd86 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:14.105228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6154d1-36a6-48fa-98ed-6a6e648fbd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:14.105500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6154d1-36a6-48fa-98ed-6a6e648fbd86 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:14.106550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6154d1-36a6-48fa-98ed-6a6e648fbd86 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:14.106667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6154d1-36a6-48fa-98ed-6a6e648fbd86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:14.107989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6154d1-36a6-48fa-98ed-6a6e648fbd86 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:14.122562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 196/400] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:50:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:14.123988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6154d1-36a6-48fa-98ed-6a6e648fbd86 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:14.126996 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:14.158535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd1eab-7d84-43a9-9148-e0e182996e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:14.159451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd1eab-7d84-43a9-9148-e0e182996e86 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:14.159835 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd1eab-7d84-43a9-9148-e0e182996e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:14.160212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd1eab-7d84-43a9-9148-e0e182996e86 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:14.190731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd1eab-7d84-43a9-9148-e0e182996e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-65cd1eab-7d84-43a9-9148-e0e182996e86 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:14.208553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:14.209757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:14.215635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 205/401] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:50:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:14.225345 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b3829-6b0a-46f2-b581-8ce547cc8438 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:14.225697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b3829-6b0a-46f2-b581-8ce547cc8438 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:14.225930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b3829-6b0a-46f2-b581-8ce547cc8438 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:14.228138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 206/402] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:50:14.228858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b3829-6b0a-46f2-b581-8ce547cc8438 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:14.250467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6154d1-36a6-48fa-98ed-6a6e648fbd86 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:14.298240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af733120-d34c-467c-b536-4ba33febf330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:14.299435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af733120-d34c-467c-b536-4ba33febf330 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:14.299984 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af733120-d34c-467c-b536-4ba33febf330 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:14.300427 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af733120-d34c-467c-b536-4ba33febf330 None admin] PATH_INFO: `/v3/groups/84dfd81bd5b04b0580f570753687e1ec` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:14.301866 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af733120-d34c-467c-b536-4ba33febf330 None admin] RBAC: Authorizing `identity:get_group(group_id=84dfd81bd5b04b0580f570753687e1ec)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:14.303516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af733120-d34c-467c-b536-4ba33febf330 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:14.305903 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 197/403] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:14 2022] GET /identity/v3/groups/84dfd81bd5b04b0580f570753687e1ec => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:14.307280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec0730-ddb4-46dc-ba8a-beedf9124eff None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:14.307669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec0730-ddb4-46dc-ba8a-beedf9124eff None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:14.308038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec0730-ddb4-46dc-ba8a-beedf9124eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:14.330772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-42ec0730-ddb4-46dc-ba8a-beedf9124eff None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:14.332404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-42ec0730-ddb4-46dc-ba8a-beedf9124eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:14.384717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b3829-6b0a-46f2-b581-8ce547cc8438 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:14.418602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:14.424206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 198/404] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:14.425146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec0730-ddb4-46dc-ba8a-beedf9124eff None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:14.432838 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d882dec-2fb3-4fe8-aa1e-de9390b7068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:14.434222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d882dec-2fb3-4fe8-aa1e-de9390b7068e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:14.434669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d882dec-2fb3-4fe8-aa1e-de9390b7068e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:14.435143 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d882dec-2fb3-4fe8-aa1e-de9390b7068e None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:14.436013 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d882dec-2fb3-4fe8-aa1e-de9390b7068e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:14.436500 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d882dec-2fb3-4fe8-aa1e-de9390b7068e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:14.445466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d882dec-2fb3-4fe8-aa1e-de9390b7068e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:14.466474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 207/405] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:14.467710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d882dec-2fb3-4fe8-aa1e-de9390b7068e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:14.482515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:14.559638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec0730-ddb4-46dc-ba8a-beedf9124eff None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:14.617470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42da4ec-6d30-4bbc-ace2-757387f2d3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:14.619254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42da4ec-6d30-4bbc-ace2-757387f2d3fd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:14.620030 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42da4ec-6d30-4bbc-ace2-757387f2d3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:14.620315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42da4ec-6d30-4bbc-ace2-757387f2d3fd None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:14.622226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42da4ec-6d30-4bbc-ace2-757387f2d3fd None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:14.624190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42da4ec-6d30-4bbc-ace2-757387f2d3fd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:14.627069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 199/406] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:14 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:14.628384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42da4ec-6d30-4bbc-ace2-757387f2d3fd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:14.655661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d882dec-2fb3-4fe8-aa1e-de9390b7068e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:14.714340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07590b63-86e7-4cb0-b1d4-5ab870b6f4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:14.715431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07590b63-86e7-4cb0-b1d4-5ab870b6f4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:14.715797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07590b63-86e7-4cb0-b1d4-5ab870b6f4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:14.716224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07590b63-86e7-4cb0-b1d4-5ab870b6f4e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:14.732897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07590b63-86e7-4cb0-b1d4-5ab870b6f4e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:14.734222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-07590b63-86e7-4cb0-b1d4-5ab870b6f4e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:14.747324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:14.748582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:14.749527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:14.749527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:14.749527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:14.749527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:14.749527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:14.749527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:14.749527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:14.749527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:14.749527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:14.749527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:14.749527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:50:14.749527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:14.750346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 208/407] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:50:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:14.756525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07590b63-86e7-4cb0-b1d4-5ab870b6f4e1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:14.790576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42da4ec-6d30-4bbc-ace2-757387f2d3fd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:14.840654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21926d-aec3-41bc-9f49-c9fbd086f1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:14.841653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21926d-aec3-41bc-9f49-c9fbd086f1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:14.841939 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21926d-aec3-41bc-9f49-c9fbd086f1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:14.842205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21926d-aec3-41bc-9f49-c9fbd086f1f7 None admin] PATH_INFO: `/v3/groups/84dfd81bd5b04b0580f570753687e1ec` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:14.843806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21926d-aec3-41bc-9f49-c9fbd086f1f7 None admin] RBAC: Authorizing `identity:get_group(group_id=84dfd81bd5b04b0580f570753687e1ec)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:14.845975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21926d-aec3-41bc-9f49-c9fbd086f1f7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:14.848682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 200/408] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:14 2022] GET /identity/v3/groups/84dfd81bd5b04b0580f570753687e1ec => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:14.849532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21926d-aec3-41bc-9f49-c9fbd086f1f7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:14.884991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:14.934538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07590b63-86e7-4cb0-b1d4-5ab870b6f4e1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:14.992391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d99c3f-dcd8-42fc-a86b-1e65d814ca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:14.993239 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d99c3f-dcd8-42fc-a86b-1e65d814ca1b None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:14.993518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d99c3f-dcd8-42fc-a86b-1e65d814ca1b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:14.993819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d99c3f-dcd8-42fc-a86b-1e65d814ca1b None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda/groups/84dfd81bd5b04b0580f570753687e1ec/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:14.996012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d99c3f-dcd8-42fc-a86b-1e65d814ca1b None admin] RBAC: Authorizing `identity:create_grant(project_id=e443cfb7bf354891a724342b87c62cda, group_id=84dfd81bd5b04b0580f570753687e1ec, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:14.998587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d99c3f-dcd8-42fc-a86b-1e65d814ca1b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:15.017416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21926d-aec3-41bc-9f49-c9fbd086f1f7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:15.026236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 209/409] 10.222.0.43 () {58 vars in 1442 bytes} [Mon Jul 4 07:50:14 2022] PUT /identity/v3/projects/e443cfb7bf354891a724342b87c62cda/groups/84dfd81bd5b04b0580f570753687e1ec/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:50:15.034439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d99c3f-dcd8-42fc-a86b-1e65d814ca1b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:15.094986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dfa8f-2359-4e1b-9a3f-4fb4dfbfc97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:15.096166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dfa8f-2359-4e1b-9a3f-4fb4dfbfc97a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:15.096800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dfa8f-2359-4e1b-9a3f-4fb4dfbfc97a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:15.097296 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dfa8f-2359-4e1b-9a3f-4fb4dfbfc97a None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:15.098743 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dfa8f-2359-4e1b-9a3f-4fb4dfbfc97a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:15.099708 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dfa8f-2359-4e1b-9a3f-4fb4dfbfc97a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:15.101549 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dfa8f-2359-4e1b-9a3f-4fb4dfbfc97a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:15.106692 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:15.126920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 201/410] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:15.140712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dfa8f-2359-4e1b-9a3f-4fb4dfbfc97a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:15.288090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d99c3f-dcd8-42fc-a86b-1e65d814ca1b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:15.346528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:15.361154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257b45a-5204-4303-9f77-ebd1e48f646e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:15.362159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257b45a-5204-4303-9f77-ebd1e48f646e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:15.362447 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257b45a-5204-4303-9f77-ebd1e48f646e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:15.362726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257b45a-5204-4303-9f77-ebd1e48f646e None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:15.365124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257b45a-5204-4303-9f77-ebd1e48f646e None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:15.367053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257b45a-5204-4303-9f77-ebd1e48f646e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:15.369565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 210/411] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:15 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:15.370907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f7dbd-7be1-44f2-aef7-dd762c1782bb None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:15.371197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f7dbd-7be1-44f2-aef7-dd762c1782bb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:15.371476 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f7dbd-7be1-44f2-aef7-dd762c1782bb None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:15.372750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 211/412] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:15.378773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f7dbd-7be1-44f2-aef7-dd762c1782bb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:15.393870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dfa8f-2359-4e1b-9a3f-4fb4dfbfc97a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:15.450093 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89288c-2fa2-4791-9bf5-273b14b88a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:15.451012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89288c-2fa2-4791-9bf5-273b14b88a2c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:15.451301 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89288c-2fa2-4791-9bf5-273b14b88a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:15.451607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89288c-2fa2-4791-9bf5-273b14b88a2c None admin] PATH_INFO: `/v3/groups/43db6823e7dd4683a5f5a2c2959214f6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:15.453904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89288c-2fa2-4791-9bf5-273b14b88a2c None admin] RBAC: Authorizing `identity:get_group(group_id=43db6823e7dd4683a5f5a2c2959214f6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:15.455345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89288c-2fa2-4791-9bf5-273b14b88a2c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:15.457581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 202/413] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:15 2022] GET /identity/v3/groups/43db6823e7dd4683a5f5a2c2959214f6 => generated 229 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:15.459200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82369619-af11-4117-810f-d38e3b356b31 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:15.459508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82369619-af11-4117-810f-d38e3b356b31 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:15.459792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82369619-af11-4117-810f-d38e3b356b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:15.480349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-82369619-af11-4117-810f-d38e3b356b31 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:15.481585 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-82369619-af11-4117-810f-d38e3b356b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:15.572345 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f7dbd-7be1-44f2-aef7-dd762c1782bb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:15.612038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:15.624637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 203/414] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:15.626297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82369619-af11-4117-810f-d38e3b356b31 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:15.655224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63486d7c-5176-47f5-b69b-c1e6decd5ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:15.657146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63486d7c-5176-47f5-b69b-c1e6decd5ad5 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:15.657662 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63486d7c-5176-47f5-b69b-c1e6decd5ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:15.658331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63486d7c-5176-47f5-b69b-c1e6decd5ad5 None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda/groups/84dfd81bd5b04b0580f570753687e1ec/roles/794ff5f5032d40a98e67fb0df718b75c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:15.668274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63486d7c-5176-47f5-b69b-c1e6decd5ad5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e443cfb7bf354891a724342b87c62cda, group_id=84dfd81bd5b04b0580f570753687e1ec, role_id=794ff5f5032d40a98e67fb0df718b75c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:15.673057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63486d7c-5176-47f5-b69b-c1e6decd5ad5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:15.712820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 212/415] 10.222.0.43 () {58 vars in 1442 bytes} [Mon Jul 4 07:50:15 2022] PUT /identity/v3/projects/e443cfb7bf354891a724342b87c62cda/groups/84dfd81bd5b04b0580f570753687e1ec/roles/794ff5f5032d40a98e67fb0df718b75c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:50:15.714730 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63486d7c-5176-47f5-b69b-c1e6decd5ad5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:15.762203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82369619-af11-4117-810f-d38e3b356b31 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:15.815019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e79b-e2b9-4db2-898f-9c3814216ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:15.816876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e79b-e2b9-4db2-898f-9c3814216ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:15.820459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e79b-e2b9-4db2-898f-9c3814216ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:15.821159 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e79b-e2b9-4db2-898f-9c3814216ae8 None admin] PATH_INFO: `/v3/projects/cf3a0790796c49f294035007b8a8c25a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:15.823247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5e79b-e2b9-4db2-898f-9c3814216ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=cf3a0790796c49f294035007b8a8c25a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:15.828717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5e79b-e2b9-4db2-898f-9c3814216ae8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:15.832048 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 204/416] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:15 2022] GET /identity/v3/projects/cf3a0790796c49f294035007b8a8c25a => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:15.841594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e79b-e2b9-4db2-898f-9c3814216ae8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:15.895533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:15.940086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:15.997252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e79b-e2b9-4db2-898f-9c3814216ae8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:16.036879 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63486d7c-5176-47f5-b69b-c1e6decd5ad5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:16.056605 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d04738-3332-4ba7-963f-fcdb32b9e91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:16.057556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d04738-3332-4ba7-963f-fcdb32b9e91b None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:16.057894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d04738-3332-4ba7-963f-fcdb32b9e91b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:16.058777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d04738-3332-4ba7-963f-fcdb32b9e91b None admin] PATH_INFO: `/v3/projects/cf3a0790796c49f294035007b8a8c25a/groups/43db6823e7dd4683a5f5a2c2959214f6/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:16.061516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d04738-3332-4ba7-963f-fcdb32b9e91b None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3a0790796c49f294035007b8a8c25a, group_id=43db6823e7dd4683a5f5a2c2959214f6, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:16.063514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d04738-3332-4ba7-963f-fcdb32b9e91b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:16.096935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 205/417] 10.222.0.43 () {58 vars in 1442 bytes} [Mon Jul 4 07:50:15 2022] PUT /identity/v3/projects/cf3a0790796c49f294035007b8a8c25a/groups/43db6823e7dd4683a5f5a2c2959214f6/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:50:16.117763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a999b-14bc-426f-a66e-8eb8ad7e2a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:16.117763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a999b-14bc-426f-a66e-8eb8ad7e2a31 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:16.117763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a999b-14bc-426f-a66e-8eb8ad7e2a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:16.117763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a999b-14bc-426f-a66e-8eb8ad7e2a31 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:16.151973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a999b-14bc-426f-a66e-8eb8ad7e2a31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ae9a999b-14bc-426f-a66e-8eb8ad7e2a31 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:16.179763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:16.180798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:16.181640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:16.181640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:16.181640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:16.181640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:16.181640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:16.181640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:16.181640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:16.181640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:16.181640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:16.181640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:16.181640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:16.181640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:16.183256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 213/418] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:50:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 469 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:16.191871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d04738-3332-4ba7-963f-fcdb32b9e91b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:16.355648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d04738-3332-4ba7-963f-fcdb32b9e91b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:16.402766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:16.424598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a94c00-362c-4f9c-8d50-d9254f1a0eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:16.425689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a94c00-362c-4f9c-8d50-d9254f1a0eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:16.426195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a94c00-362c-4f9c-8d50-d9254f1a0eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:16.426693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a94c00-362c-4f9c-8d50-d9254f1a0eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:16.428133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a94c00-362c-4f9c-8d50-d9254f1a0eb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:16.428692 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a94c00-362c-4f9c-8d50-d9254f1a0eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:16.431521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a94c00-362c-4f9c-8d50-d9254f1a0eb1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:16.451236 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 206/419] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:16.456993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a999b-14bc-426f-a66e-8eb8ad7e2a31 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:16.660022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a999b-14bc-426f-a66e-8eb8ad7e2a31 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:16.693652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:16.725342 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511e960-a138-4320-ab19-422b231025a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:16.726124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e960-a138-4320-ab19-422b231025a6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:16.726479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e960-a138-4320-ab19-422b231025a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:16.726971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e960-a138-4320-ab19-422b231025a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:16.806905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511e960-a138-4320-ab19-422b231025a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:16.808400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511e960-a138-4320-ab19-422b231025a6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:16.859193 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273ea0c-a928-4253-9a14-9fe66d0f215b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:16.859528 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273ea0c-a928-4253-9a14-9fe66d0f215b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:16.859746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273ea0c-a928-4253-9a14-9fe66d0f215b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:16.861984 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 207/420] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:16.868039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a965d-3a6b-48d5-a556-644e4c9094ad None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:16.868633 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a965d-3a6b-48d5-a556-644e4c9094ad None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:16.869135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a965d-3a6b-48d5-a556-644e4c9094ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:16.887861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c40a965d-3a6b-48d5-a556-644e4c9094ad None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:16.889438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c40a965d-3a6b-48d5-a556-644e4c9094ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:16.946106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 214/421] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 07:50:16.957672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511e960-a138-4320-ab19-422b231025a6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:16.978050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:17.007676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 208/422] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:17.032485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a965d-3a6b-48d5-a556-644e4c9094ad None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:17.127169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511e960-a138-4320-ab19-422b231025a6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:17.192146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a965d-3a6b-48d5-a556-644e4c9094ad None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:17.198289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dacb9a-a90d-491f-94f3-527d78ae8510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:17.199820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dacb9a-a90d-491f-94f3-527d78ae8510 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:17.200689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dacb9a-a90d-491f-94f3-527d78ae8510 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:17.202552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dacb9a-a90d-491f-94f3-527d78ae8510 None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:17.205581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dacb9a-a90d-491f-94f3-527d78ae8510 None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:17.217888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dacb9a-a90d-491f-94f3-527d78ae8510 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:17.221401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 215/423] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:16 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:17.228295 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dacb9a-a90d-491f-94f3-527d78ae8510 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:17.252090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:17.315401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dcebd-b0c8-4244-b3e2-20f50789779f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:17.316317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dcebd-b0c8-4244-b3e2-20f50789779f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:17.316597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dcebd-b0c8-4244-b3e2-20f50789779f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:17.316880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dcebd-b0c8-4244-b3e2-20f50789779f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:17.344826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293dcebd-b0c8-4244-b3e2-20f50789779f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-293dcebd-b0c8-4244-b3e2-20f50789779f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:17.361356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:17.362396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:17.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:17.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:17.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:17.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:17.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:17.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:17.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:17.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:17.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:17.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:17.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:17.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:17.368592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 209/424] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:17.371278 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dcebd-b0c8-4244-b3e2-20f50789779f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:17.402318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:17.418774 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dacb9a-a90d-491f-94f3-527d78ae8510 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:17.471574 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bc795-2411-4c09-bab5-b18cadaba08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:17.472467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bc795-2411-4c09-bab5-b18cadaba08e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:17.472749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bc795-2411-4c09-bab5-b18cadaba08e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:17.473026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bc795-2411-4c09-bab5-b18cadaba08e None admin] PATH_INFO: `/v3/groups/84dfd81bd5b04b0580f570753687e1ec` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:17.475649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bc795-2411-4c09-bab5-b18cadaba08e None admin] RBAC: Authorizing `identity:get_group(group_id=84dfd81bd5b04b0580f570753687e1ec)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:17.477187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bc795-2411-4c09-bab5-b18cadaba08e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:17.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 216/425] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:17 2022] GET /identity/v3/groups/84dfd81bd5b04b0580f570753687e1ec => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:17.486570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bc795-2411-4c09-bab5-b18cadaba08e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:17.595031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dcebd-b0c8-4244-b3e2-20f50789779f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:17.659558 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11abed5-59f9-4db9-ae32-6aae6ff9b7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:17.660525 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11abed5-59f9-4db9-ae32-6aae6ff9b7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:17.660819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11abed5-59f9-4db9-ae32-6aae6ff9b7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:17.661099 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11abed5-59f9-4db9-ae32-6aae6ff9b7f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:17.661891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11abed5-59f9-4db9-ae32-6aae6ff9b7f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:17.662201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11abed5-59f9-4db9-ae32-6aae6ff9b7f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:17.663789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11abed5-59f9-4db9-ae32-6aae6ff9b7f0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:17.674737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 210/426] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:50:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:17.680327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11abed5-59f9-4db9-ae32-6aae6ff9b7f0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:17.714655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bc795-2411-4c09-bab5-b18cadaba08e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:17.747801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:17.803948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271f845-4696-4128-967e-c65b277d6897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:17.804681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271f845-4696-4128-967e-c65b277d6897 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:17.804962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271f845-4696-4128-967e-c65b277d6897 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:17.805236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271f845-4696-4128-967e-c65b277d6897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:17.806134 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271f845-4696-4128-967e-c65b277d6897 None admin] RBAC: Adding query filter params (group.id=84dfd81bd5b04b0580f570753687e1ec, role.id=794ff5f5032d40a98e67fb0df718b75c, scope.project.id=d44ad181a25048dea48323087ad2f3ca) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:17.806443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271f845-4696-4128-967e-c65b277d6897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:17.807991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271f845-4696-4128-967e-c65b277d6897 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:17.828419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 217/427] 10.222.0.43 () {56 vars in 1376 bytes} [Mon Jul 4 07:50:17 2022] GET /identity/v3/role_assignments?group.id=84dfd81bd5b04b0580f570753687e1ec&scope.project.id=d44ad181a25048dea48323087ad2f3ca&role.id=794ff5f5032d40a98e67fb0df718b75c => generated 611 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:17.852590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11abed5-59f9-4db9-ae32-6aae6ff9b7f0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:17.930709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b7182-26b0-4371-b167-2434fa8a8950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:17.931412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b7182-26b0-4371-b167-2434fa8a8950 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:17.931688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b7182-26b0-4371-b167-2434fa8a8950 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:17.931967 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b7182-26b0-4371-b167-2434fa8a8950 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:17.978597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0410b-927e-4504-a799-23fd5b0ecd0e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:17.978877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0410b-927e-4504-a799-23fd5b0ecd0e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:17.979265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0410b-927e-4504-a799-23fd5b0ecd0e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:17.980924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 218/428] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:17.986754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391515d-0330-4165-9df4-8b8e18d58bb7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:17.987153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391515d-0330-4165-9df4-8b8e18d58bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:17.987338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391515d-0330-4165-9df4-8b8e18d58bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:18.014302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c391515d-0330-4165-9df4-8b8e18d58bb7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:18.018088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c391515d-0330-4165-9df4-8b8e18d58bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:18.042270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:18.050328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b7182-26b0-4371-b167-2434fa8a8950 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:18.055898 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b7182-26b0-4371-b167-2434fa8a8950 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:18.155964 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:18.167463 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 219/429] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:50:18.191881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391515d-0330-4165-9df4-8b8e18d58bb7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:18.280237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 211/430] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 07:50:18.289886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b7182-26b0-4371-b167-2434fa8a8950 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:18.352534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391515d-0330-4165-9df4-8b8e18d58bb7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:18.397057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390594b-2433-49b6-b775-204efcfceca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:18.398521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390594b-2433-49b6-b775-204efcfceca1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:18.398682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390594b-2433-49b6-b775-204efcfceca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:18.398873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390594b-2433-49b6-b775-204efcfceca1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:18.413434 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390594b-2433-49b6-b775-204efcfceca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d390594b-2433-49b6-b775-204efcfceca1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:18.431750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:18.432765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:18.433595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 220/431] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:18.439479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087db2d0-a4ab-4989-ae92-50f67be6cda1 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:18.439800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087db2d0-a4ab-4989-ae92-50f67be6cda1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:18.440072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087db2d0-a4ab-4989-ae92-50f67be6cda1 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:18.441569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b7182-26b0-4371-b167-2434fa8a8950 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:18.441910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 221/432] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:18.443881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087db2d0-a4ab-4989-ae92-50f67be6cda1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:18.454788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:18.484954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3cbf8-b01b-450e-bbe1-62e5fcf3b24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:18.486328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3cbf8-b01b-450e-bbe1-62e5fcf3b24a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:18.486822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3cbf8-b01b-450e-bbe1-62e5fcf3b24a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:18.487301 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3cbf8-b01b-450e-bbe1-62e5fcf3b24a None admin] PATH_INFO: `/v3/projects/d44ad181a25048dea48323087ad2f3ca` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:18.489055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3cbf8-b01b-450e-bbe1-62e5fcf3b24a None admin] RBAC: Authorizing `identity:get_project(project_id=d44ad181a25048dea48323087ad2f3ca)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:18.490651 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3cbf8-b01b-450e-bbe1-62e5fcf3b24a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:18.497520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 212/433] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:18 2022] GET /identity/v3/projects/d44ad181a25048dea48323087ad2f3ca => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:18.499002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb98e04-237f-4df5-b3bf-198b6cf085b4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:18.499326 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb98e04-237f-4df5-b3bf-198b6cf085b4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:18.499636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb98e04-237f-4df5-b3bf-198b6cf085b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:18.512447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:18.520278 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5bb98e04-237f-4df5-b3bf-198b6cf085b4 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:18.521544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5bb98e04-237f-4df5-b3bf-198b6cf085b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:18.575252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087db2d0-a4ab-4989-ae92-50f67be6cda1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:18.602582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 213/434] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:18.610849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb98e04-237f-4df5-b3bf-198b6cf085b4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:18.627115 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87209c6e-6cb9-4aa2-a09c-672c504338c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:18.628549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87209c6e-6cb9-4aa2-a09c-672c504338c3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:18.629323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87209c6e-6cb9-4aa2-a09c-672c504338c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:18.630010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87209c6e-6cb9-4aa2-a09c-672c504338c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:18.631240 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87209c6e-6cb9-4aa2-a09c-672c504338c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:18.631710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87209c6e-6cb9-4aa2-a09c-672c504338c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:18.634084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87209c6e-6cb9-4aa2-a09c-672c504338c3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:18.651685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 222/435] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:50:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:18.652606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87209c6e-6cb9-4aa2-a09c-672c504338c3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:18.739143 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:18.760244 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb98e04-237f-4df5-b3bf-198b6cf085b4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:18.782287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87209c6e-6cb9-4aa2-a09c-672c504338c3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:18.814427 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce05dfd-2eff-42f2-9f3b-947794748c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:18.815712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce05dfd-2eff-42f2-9f3b-947794748c0a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:18.816209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce05dfd-2eff-42f2-9f3b-947794748c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:18.816693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce05dfd-2eff-42f2-9f3b-947794748c0a None admin] PATH_INFO: `/v3/groups/84dfd81bd5b04b0580f570753687e1ec` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:18.818767 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce05dfd-2eff-42f2-9f3b-947794748c0a None admin] RBAC: Authorizing `identity:get_group(group_id=84dfd81bd5b04b0580f570753687e1ec)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:18.821411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce05dfd-2eff-42f2-9f3b-947794748c0a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:18.822055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381ba7f-26b1-4386-a88d-ec3bad1c10b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:18.822890 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381ba7f-26b1-4386-a88d-ec3bad1c10b9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:18.823170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381ba7f-26b1-4386-a88d-ec3bad1c10b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:18.823445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381ba7f-26b1-4386-a88d-ec3bad1c10b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:18.830062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 214/436] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:18 2022] GET /identity/v3/groups/84dfd81bd5b04b0580f570753687e1ec => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:18.831366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce05dfd-2eff-42f2-9f3b-947794748c0a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:18.836257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381ba7f-26b1-4386-a88d-ec3bad1c10b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:18.839713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5381ba7f-26b1-4386-a88d-ec3bad1c10b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:18.846037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:18.847075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 223/437] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:50:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:18.853664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381ba7f-26b1-4386-a88d-ec3bad1c10b9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:18.969029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381ba7f-26b1-4386-a88d-ec3bad1c10b9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:18.972496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce05dfd-2eff-42f2-9f3b-947794748c0a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:19.002028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:19.011685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619c30c-65bb-413b-ba23-c618ce4e3615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:19.012332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c30c-65bb-413b-ba23-c618ce4e3615 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:19.012600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c30c-65bb-413b-ba23-c618ce4e3615 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:19.012907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c30c-65bb-413b-ba23-c618ce4e3615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:19.013066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54b941-7505-4603-b3be-b1c2b0475601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:19.013781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54b941-7505-4603-b3be-b1c2b0475601 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:19.014066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54b941-7505-4603-b3be-b1c2b0475601 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:19.014339 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54b941-7505-4603-b3be-b1c2b0475601 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:19.015186 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54b941-7505-4603-b3be-b1c2b0475601 None admin] RBAC: Adding query filter params (group.id=84dfd81bd5b04b0580f570753687e1ec, role.id=3edce31396af478ea1ea7b76d33da530, scope.project.id=d44ad181a25048dea48323087ad2f3ca) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:19.015503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54b941-7505-4603-b3be-b1c2b0475601 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:19.016978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54b941-7505-4603-b3be-b1c2b0475601 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:19.023848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 224/438] 10.222.0.43 () {56 vars in 1376 bytes} [Mon Jul 4 07:50:18 2022] GET /identity/v3/role_assignments?group.id=84dfd81bd5b04b0580f570753687e1ec&scope.project.id=d44ad181a25048dea48323087ad2f3ca&role.id=3edce31396af478ea1ea7b76d33da530 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:19.024728 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54b941-7505-4603-b3be-b1c2b0475601 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:19.124187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619c30c-65bb-413b-ba23-c618ce4e3615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:19.126690 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619c30c-65bb-413b-ba23-c618ce4e3615 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:19.130607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:19.198688 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54b941-7505-4603-b3be-b1c2b0475601 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:19.244559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e451a-3b35-41ce-bf18-eec998143985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:19.245431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e451a-3b35-41ce-bf18-eec998143985 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:19.245750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e451a-3b35-41ce-bf18-eec998143985 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:19.246033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e451a-3b35-41ce-bf18-eec998143985 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:19.246794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e451a-3b35-41ce-bf18-eec998143985 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:19.247107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e451a-3b35-41ce-bf18-eec998143985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:19.248535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e451a-3b35-41ce-bf18-eec998143985 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:19.261885 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 225/439] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:19.264539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae91f7-0325-457b-bfcd-fe8082eb954c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:19.264924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae91f7-0325-457b-bfcd-fe8082eb954c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:19.265422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae91f7-0325-457b-bfcd-fe8082eb954c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:19.267446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 226/440] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:19.268706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae91f7-0325-457b-bfcd-fe8082eb954c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:19.277342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 215/441] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:19.280008 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a4ba9-4e24-45b2-a8b2-269e684eb8ff None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:19.280402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a4ba9-4e24-45b2-a8b2-269e684eb8ff None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:19.280681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a4ba9-4e24-45b2-a8b2-269e684eb8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:19.301408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b05a4ba9-4e24-45b2-a8b2-269e684eb8ff None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:19.303214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b05a4ba9-4e24-45b2-a8b2-269e684eb8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:19.364773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:19.389062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 216/442] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:19.390064 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a4ba9-4e24-45b2-a8b2-269e684eb8ff None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:19.435895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae91f7-0325-457b-bfcd-fe8082eb954c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:19.443711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:19.482446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac4b64-1cac-4c22-9c5b-2e88a3488b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:19.483138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac4b64-1cac-4c22-9c5b-2e88a3488b0d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:19.483419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac4b64-1cac-4c22-9c5b-2e88a3488b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:19.483690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac4b64-1cac-4c22-9c5b-2e88a3488b0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:19.543493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a4ba9-4e24-45b2-a8b2-269e684eb8ff None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:19.590578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac4b64-1cac-4c22-9c5b-2e88a3488b0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:19.592139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac4b64-1cac-4c22-9c5b-2e88a3488b0d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:19.603357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c64fd6-ee01-444e-a33d-445f048c2743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:19.605125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c64fd6-ee01-444e-a33d-445f048c2743 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:19.605479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c64fd6-ee01-444e-a33d-445f048c2743 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:19.607045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c64fd6-ee01-444e-a33d-445f048c2743 None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:19.608486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c64fd6-ee01-444e-a33d-445f048c2743 None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:19.610108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c64fd6-ee01-444e-a33d-445f048c2743 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:19.612512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 217/443] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:19 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:19.612799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c64fd6-ee01-444e-a33d-445f048c2743 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:19.673612 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:19.726786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 227/444] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:19.727944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac4b64-1cac-4c22-9c5b-2e88a3488b0d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:19.760326 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c64fd6-ee01-444e-a33d-445f048c2743 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:19.762120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:19.798736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c151b5-1a8a-44d6-8840-683f7b882fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:19.800014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c151b5-1a8a-44d6-8840-683f7b882fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:19.800454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c151b5-1a8a-44d6-8840-683f7b882fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:19.800906 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c151b5-1a8a-44d6-8840-683f7b882fb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:19.816789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c151b5-1a8a-44d6-8840-683f7b882fb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-99c151b5-1a8a-44d6-8840-683f7b882fb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:19.830644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:19.831614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:19.832419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:19.832419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:19.832419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:19.832419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:19.832419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:19.832419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:19.832419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:19.832419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:19.832419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:19.832419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:19.832419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:50:19.832419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:19.833038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 218/445] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:50:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:19.833774 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c151b5-1a8a-44d6-8840-683f7b882fb2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:19.858147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac4b64-1cac-4c22-9c5b-2e88a3488b0d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:19.895298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfac6c-083f-44ea-9e92-4aa6de0637b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:19.896539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfac6c-083f-44ea-9e92-4aa6de0637b1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:19.896887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfac6c-083f-44ea-9e92-4aa6de0637b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:19.897324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfac6c-083f-44ea-9e92-4aa6de0637b1 None admin] PATH_INFO: `/v3/groups/84dfd81bd5b04b0580f570753687e1ec` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:19.899166 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfac6c-083f-44ea-9e92-4aa6de0637b1 None admin] RBAC: Authorizing `identity:get_group(group_id=84dfd81bd5b04b0580f570753687e1ec)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:19.901137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfac6c-083f-44ea-9e92-4aa6de0637b1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:19.903218 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 228/446] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:19 2022] GET /identity/v3/groups/84dfd81bd5b04b0580f570753687e1ec => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:19.904389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfac6c-083f-44ea-9e92-4aa6de0637b1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:19.931718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:19.943460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c151b5-1a8a-44d6-8840-683f7b882fb2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:19.986012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858633f-ff07-450a-b662-b8974f7f953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:19.986892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858633f-ff07-450a-b662-b8974f7f953e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:19.987168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858633f-ff07-450a-b662-b8974f7f953e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:19.987439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858633f-ff07-450a-b662-b8974f7f953e None admin] PATH_INFO: `/v3/projects/e443cfb7bf354891a724342b87c62cda` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:19.988649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858633f-ff07-450a-b662-b8974f7f953e None admin] RBAC: Authorizing `identity:get_project(project_id=e443cfb7bf354891a724342b87c62cda)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:19.990133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858633f-ff07-450a-b662-b8974f7f953e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:19.991912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 219/447] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:19 2022] GET /identity/v3/projects/e443cfb7bf354891a724342b87c62cda => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:19.992908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858633f-ff07-450a-b662-b8974f7f953e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:20.022724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfac6c-083f-44ea-9e92-4aa6de0637b1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:20.026462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:20.059961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a7a24-955e-40ec-a576-65e495894f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:20.060653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a7a24-955e-40ec-a576-65e495894f29 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:20.060932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a7a24-955e-40ec-a576-65e495894f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:20.061207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a7a24-955e-40ec-a576-65e495894f29 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:20.062015 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a7a24-955e-40ec-a576-65e495894f29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:20.062315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a7a24-955e-40ec-a576-65e495894f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:20.063720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a7a24-955e-40ec-a576-65e495894f29 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:20.072765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 229/448] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:20.073599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a7a24-955e-40ec-a576-65e495894f29 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:20.110552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858633f-ff07-450a-b662-b8974f7f953e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:20.151271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912cb65-a1de-4f5c-b723-039008563020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:20.151985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912cb65-a1de-4f5c-b723-039008563020 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:20.152271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912cb65-a1de-4f5c-b723-039008563020 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:20.152676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912cb65-a1de-4f5c-b723-039008563020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:20.153400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912cb65-a1de-4f5c-b723-039008563020 None admin] RBAC: Adding query filter params (group.id=84dfd81bd5b04b0580f570753687e1ec, role.id=3edce31396af478ea1ea7b76d33da530, scope.project.id=e443cfb7bf354891a724342b87c62cda) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:20.154260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912cb65-a1de-4f5c-b723-039008563020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:20.155205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912cb65-a1de-4f5c-b723-039008563020 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:20.162044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 220/449] 10.222.0.43 () {56 vars in 1376 bytes} [Mon Jul 4 07:50:19 2022] GET /identity/v3/role_assignments?group.id=84dfd81bd5b04b0580f570753687e1ec&scope.project.id=e443cfb7bf354891a724342b87c62cda&role.id=3edce31396af478ea1ea7b76d33da530 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:20.162893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912cb65-a1de-4f5c-b723-039008563020 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:20.177603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:20.206219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a7a24-955e-40ec-a576-65e495894f29 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:20.273461 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756bf8f-9fda-4c69-8a44-634e73df1d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:20.276141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756bf8f-9fda-4c69-8a44-634e73df1d48 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:20.276661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756bf8f-9fda-4c69-8a44-634e73df1d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:20.277571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756bf8f-9fda-4c69-8a44-634e73df1d48 None admin] PATH_INFO: `/v3/groups/84dfd81bd5b04b0580f570753687e1ec` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:20.280647 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756bf8f-9fda-4c69-8a44-634e73df1d48 None admin] RBAC: Authorizing `identity:get_group(group_id=84dfd81bd5b04b0580f570753687e1ec)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:20.285087 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756bf8f-9fda-4c69-8a44-634e73df1d48 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:20.289516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 230/450] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:20 2022] GET /identity/v3/groups/84dfd81bd5b04b0580f570753687e1ec => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:20.296373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:20.299138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756bf8f-9fda-4c69-8a44-634e73df1d48 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:20.306644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912cb65-a1de-4f5c-b723-039008563020 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:20.355152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fd85d-2d9f-4636-94e2-b1b22bdd9a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:20.355833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fd85d-2d9f-4636-94e2-b1b22bdd9a42 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:20.356118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fd85d-2d9f-4636-94e2-b1b22bdd9a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:20.356397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fd85d-2d9f-4636-94e2-b1b22bdd9a42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:20.457910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756bf8f-9fda-4c69-8a44-634e73df1d48 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:20.466501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fd85d-2d9f-4636-94e2-b1b22bdd9a42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:20.469027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fd85d-2d9f-4636-94e2-b1b22bdd9a42 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:20.502979 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdf100-984e-47c2-a443-c88342479ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:20.503658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdf100-984e-47c2-a443-c88342479ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:20.504270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdf100-984e-47c2-a443-c88342479ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:20.504558 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdf100-984e-47c2-a443-c88342479ee9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:20.505415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdf100-984e-47c2-a443-c88342479ee9 None admin] RBAC: Adding query filter params (group.id=84dfd81bd5b04b0580f570753687e1ec, role.id=794ff5f5032d40a98e67fb0df718b75c, scope.project.id=e443cfb7bf354891a724342b87c62cda) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:20.505791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdf100-984e-47c2-a443-c88342479ee9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:20.507338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdf100-984e-47c2-a443-c88342479ee9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:20.514313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 231/451] 10.222.0.43 () {56 vars in 1376 bytes} [Mon Jul 4 07:50:20 2022] GET /identity/v3/role_assignments?group.id=84dfd81bd5b04b0580f570753687e1ec&scope.project.id=e443cfb7bf354891a724342b87c62cda&role.id=794ff5f5032d40a98e67fb0df718b75c => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:20.608061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:20.625223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 221/452] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:20.634380 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdf100-984e-47c2-a443-c88342479ee9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:20.657592 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:20.774400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdf100-984e-47c2-a443-c88342479ee9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:20.815846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8d681-6621-4731-9094-73c79fb55746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:20.816754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8d681-6621-4731-9094-73c79fb55746 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:20.817043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8d681-6621-4731-9094-73c79fb55746 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:20.817323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8d681-6621-4731-9094-73c79fb55746 None admin] PATH_INFO: `/v3/projects/cf3a0790796c49f294035007b8a8c25a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:20.818633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8d681-6621-4731-9094-73c79fb55746 None admin] RBAC: Authorizing `identity:get_project(project_id=cf3a0790796c49f294035007b8a8c25a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:20.820675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8d681-6621-4731-9094-73c79fb55746 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:20.822939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 232/453] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:50:20 2022] GET /identity/v3/projects/cf3a0790796c49f294035007b8a8c25a => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:20.829802 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fd85d-2d9f-4636-94e2-b1b22bdd9a42 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:20.959567 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fd85d-2d9f-4636-94e2-b1b22bdd9a42 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:20.998009 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2f070-85b3-4f9e-8e9f-22f807e18eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:20.998885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2f070-85b3-4f9e-8e9f-22f807e18eac None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:20.999162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2f070-85b3-4f9e-8e9f-22f807e18eac None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:20.999437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2f070-85b3-4f9e-8e9f-22f807e18eac None admin] PATH_INFO: `/v3/groups/43db6823e7dd4683a5f5a2c2959214f6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:21.000678 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2f070-85b3-4f9e-8e9f-22f807e18eac None admin] RBAC: Authorizing `identity:get_group(group_id=43db6823e7dd4683a5f5a2c2959214f6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:21.002185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2f070-85b3-4f9e-8e9f-22f807e18eac None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:21.003840 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 222/454] 10.222.0.43 () {56 vars in 1180 bytes} [Mon Jul 4 07:50:20 2022] GET /identity/v3/groups/43db6823e7dd4683a5f5a2c2959214f6 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:21.011099 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8d681-6621-4731-9094-73c79fb55746 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:21.106531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:21.127208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8d681-6621-4731-9094-73c79fb55746 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:21.171509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a9393-e035-4d02-a85c-f7f3f0a9b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:21.172204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a9393-e035-4d02-a85c-f7f3f0a9b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:21.172491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a9393-e035-4d02-a85c-f7f3f0a9b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:21.172770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a9393-e035-4d02-a85c-f7f3f0a9b7c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:21.173661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a9393-e035-4d02-a85c-f7f3f0a9b7c3 None admin] RBAC: Adding query filter params (group.id=43db6823e7dd4683a5f5a2c2959214f6, role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=cf3a0790796c49f294035007b8a8c25a) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:21.173978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a9393-e035-4d02-a85c-f7f3f0a9b7c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:21.175522 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a9393-e035-4d02-a85c-f7f3f0a9b7c3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:21.183337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 233/455] 10.222.0.43 () {56 vars in 1376 bytes} [Mon Jul 4 07:50:21 2022] GET /identity/v3/role_assignments?group.id=43db6823e7dd4683a5f5a2c2959214f6&scope.project.id=cf3a0790796c49f294035007b8a8c25a&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:23.842250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f28c8-18d8-4376-969b-99f71af401df None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:23.843072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f28c8-18d8-4376-969b-99f71af401df None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:23.843072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f28c8-18d8-4376-969b-99f71af401df None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:23.844952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 223/456] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:23.856310 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed6ef9-46a7-4567-baba-71a6a1ccaf37 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:23.856817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed6ef9-46a7-4567-baba-71a6a1ccaf37 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:23.857307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed6ef9-46a7-4567-baba-71a6a1ccaf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:23.902365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d3ed6ef9-46a7-4567-baba-71a6a1ccaf37 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:23.904491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d3ed6ef9-46a7-4567-baba-71a6a1ccaf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:24.077549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 234/457] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:24.130061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91728a09-f1b1-41ef-b752-1712bc868b56 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:24.130573 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91728a09-f1b1-41ef-b752-1712bc868b56 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:24.131082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91728a09-f1b1-41ef-b752-1712bc868b56 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:24.134448 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 224/458] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:24.143542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91728a09-f1b1-41ef-b752-1712bc868b56 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:24.143775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bec7d7-be19-4924-9291-cb5209ab63c3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:24.144264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bec7d7-be19-4924-9291-cb5209ab63c3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:24.144763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bec7d7-be19-4924-9291-cb5209ab63c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:24.191824 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f2bec7d7-be19-4924-9291-cb5209ab63c3 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:24.193051 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f2bec7d7-be19-4924-9291-cb5209ab63c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:24.290562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 235/459] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:24.290562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9fbbf-c7dc-49b7-9041-ec0a78f22c69 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:24.290562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9fbbf-c7dc-49b7-9041-ec0a78f22c69 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:24.290562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9fbbf-c7dc-49b7-9041-ec0a78f22c69 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:24.290562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 236/460] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:24.297828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46d7cd-a324-4b46-badf-2887d7748618 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:24.297828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46d7cd-a324-4b46-badf-2887d7748618 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:24.297828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46d7cd-a324-4b46-badf-2887d7748618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:24.335201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ac46d7cd-a324-4b46-badf-2887d7748618 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:24.337300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ac46d7cd-a324-4b46-badf-2887d7748618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:24.339850 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:24.346071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:24.405930 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91728a09-f1b1-41ef-b752-1712bc868b56 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:24.446150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643802f6-115e-43b3-9b6e-a4e266899263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:24.447763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643802f6-115e-43b3-9b6e-a4e266899263 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:24.448424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643802f6-115e-43b3-9b6e-a4e266899263 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:24.449069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643802f6-115e-43b3-9b6e-a4e266899263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:24.463406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643802f6-115e-43b3-9b6e-a4e266899263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:24.465183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643802f6-115e-43b3-9b6e-a4e266899263 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:24.473697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 237/461] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:24.474887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46d7cd-a324-4b46-badf-2887d7748618 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-643802f6-115e-43b3-9b6e-a4e266899263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:24.486880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:24.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:24.488709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:24.488709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:24.488709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:24.488709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:24.491215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 225/462] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:24.503382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643802f6-115e-43b3-9b6e-a4e266899263 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:24.715003 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46d7cd-a324-4b46-badf-2887d7748618 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:24.719372 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643802f6-115e-43b3-9b6e-a4e266899263 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:24.754304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:24.767235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a0f98-220f-4881-8e11-113f6565266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:24.768447 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a0f98-220f-4881-8e11-113f6565266b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:24.768939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a0f98-220f-4881-8e11-113f6565266b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:24.769423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a0f98-220f-4881-8e11-113f6565266b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:24.779206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:24.787225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a0f98-220f-4881-8e11-113f6565266b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:24.791115 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a0f98-220f-4881-8e11-113f6565266b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:24.795213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a861d5-5921-4e42-9118-cba067f32bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:24.796191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a861d5-5921-4e42-9118-cba067f32bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:24.796719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a861d5-5921-4e42-9118-cba067f32bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:24.797191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a861d5-5921-4e42-9118-cba067f32bb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:24.798153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a861d5-5921-4e42-9118-cba067f32bb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:24.798659 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a861d5-5921-4e42-9118-cba067f32bb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:24.800325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a861d5-5921-4e42-9118-cba067f32bb7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:24.817258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 226/463] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:24.818972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a861d5-5921-4e42-9118-cba067f32bb7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0a2a0f98-220f-4881-8e11-113f6565266b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:24.835081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:24.836528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:24.837980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:24.837980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:24.837980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:24.837980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:24.841392 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 238/464] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:24.843437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a0f98-220f-4881-8e11-113f6565266b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:24.961090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a861d5-5921-4e42-9118-cba067f32bb7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:25.036128 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a0f98-220f-4881-8e11-113f6565266b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:25.052022 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98b60a-8503-4514-8be7-8829e63b9930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:25.054212 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98b60a-8503-4514-8be7-8829e63b9930 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:25.054720 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98b60a-8503-4514-8be7-8829e63b9930 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:25.055052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98b60a-8503-4514-8be7-8829e63b9930 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:25.084028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98b60a-8503-4514-8be7-8829e63b9930 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:25.087299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98b60a-8503-4514-8be7-8829e63b9930 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:25.093839 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d906ac9-138b-4032-945f-9bfac3cc8e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:25.095192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d906ac9-138b-4032-945f-9bfac3cc8e22 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:25.095699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d906ac9-138b-4032-945f-9bfac3cc8e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:25.096182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d906ac9-138b-4032-945f-9bfac3cc8e22 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:25.097533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d906ac9-138b-4032-945f-9bfac3cc8e22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:25.099451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d906ac9-138b-4032-945f-9bfac3cc8e22 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0f98b60a-8503-4514-8be7-8829e63b9930 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:25.132685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:25.134443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:25.135606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:25.135606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:25.135606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:25.135606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:25.141795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 227/465] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:25.142945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98b60a-8503-4514-8be7-8829e63b9930 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:25.149157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 239/466] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:50:24 2022] POST /identity/v3/users => generated 246 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:50:25.152008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d906ac9-138b-4032-945f-9bfac3cc8e22 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:25.218999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:25.246773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:25.350648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d906ac9-138b-4032-945f-9bfac3cc8e22 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:25.360511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98b60a-8503-4514-8be7-8829e63b9930 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:25.403757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749dae6-b866-49ba-94fd-8d192d8c56bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:25.404460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749dae6-b866-49ba-94fd-8d192d8c56bd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:25.404748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749dae6-b866-49ba-94fd-8d192d8c56bd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:25.405036 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749dae6-b866-49ba-94fd-8d192d8c56bd None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:25.405839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749dae6-b866-49ba-94fd-8d192d8c56bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:25.406156 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749dae6-b866-49ba-94fd-8d192d8c56bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:25.407758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749dae6-b866-49ba-94fd-8d192d8c56bd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:25.421258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 228/467] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:25.422751 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72df186-bea1-4d8a-ac37-c28cbb9e6ed3 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:25.423067 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72df186-bea1-4d8a-ac37-c28cbb9e6ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:25.423298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676c1e-b66f-4384-9328-7425161096de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:25.423449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72df186-bea1-4d8a-ac37-c28cbb9e6ed3 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:25.424913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676c1e-b66f-4384-9328-7425161096de None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:25.427319 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676c1e-b66f-4384-9328-7425161096de None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:25.428316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676c1e-b66f-4384-9328-7425161096de None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:25.429138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 229/468] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:25.430095 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72df186-bea1-4d8a-ac37-c28cbb9e6ed3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:25.430768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676c1e-b66f-4384-9328-7425161096de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:25.431324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676c1e-b66f-4384-9328-7425161096de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:25.433529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676c1e-b66f-4384-9328-7425161096de None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:25.457167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 240/469] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:25.461592 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5582acb-45b7-4144-b302-6d92b87103f7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:25.462004 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5582acb-45b7-4144-b302-6d92b87103f7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:25.462292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5582acb-45b7-4144-b302-6d92b87103f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:25.491705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b5582acb-45b7-4144-b302-6d92b87103f7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:25.492258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b5582acb-45b7-4144-b302-6d92b87103f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:25.572439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:25.589402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 241/470] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:25.590718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5582acb-45b7-4144-b302-6d92b87103f7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:25.595795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72df186-bea1-4d8a-ac37-c28cbb9e6ed3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:25.633925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:25.638312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6e71-9452-4c83-8285-4cc28d3692d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:25.639346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6e71-9452-4c83-8285-4cc28d3692d1 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:25.639992 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6e71-9452-4c83-8285-4cc28d3692d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:25.640690 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6e71-9452-4c83-8285-4cc28d3692d1 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:25.641967 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b6e71-9452-4c83-8285-4cc28d3692d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:25.643712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b6e71-9452-4c83-8285-4cc28d3692d1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:25.678172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 230/471] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:50:25 2022] POST /identity/v3/users => generated 248 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:50:25.680649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6e71-9452-4c83-8285-4cc28d3692d1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:25.726310 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5582acb-45b7-4144-b302-6d92b87103f7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:25.793484 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110fdb5-172c-48ea-9ebf-540bf1939365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:25.794824 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110fdb5-172c-48ea-9ebf-540bf1939365 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:25.795608 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110fdb5-172c-48ea-9ebf-540bf1939365 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:25.795845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110fdb5-172c-48ea-9ebf-540bf1939365 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:25.797127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110fdb5-172c-48ea-9ebf-540bf1939365 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:25.800029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110fdb5-172c-48ea-9ebf-540bf1939365 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:25.815856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6e71-9452-4c83-8285-4cc28d3692d1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:25.844280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 242/472] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:50:25 2022] POST /identity/v3/users => generated 248 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:50:25.844280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b504a-a931-4153-99c6-6d6380c4eeb1 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:25.844280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b504a-a931-4153-99c6-6d6380c4eeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:25.844280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b504a-a931-4153-99c6-6d6380c4eeb1 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:25.844280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 243/473] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:25.857520 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49897f-32a6-4f7b-85eb-87b9ff5c1209 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:25.858513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49897f-32a6-4f7b-85eb-87b9ff5c1209 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:25.858624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49897f-32a6-4f7b-85eb-87b9ff5c1209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:25.871127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafc063-0b21-478f-af8b-ede8d76576f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:25.872361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafc063-0b21-478f-af8b-ede8d76576f8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:25.872862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafc063-0b21-478f-af8b-ede8d76576f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:25.873391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafc063-0b21-478f-af8b-ede8d76576f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:25.886829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafc063-0b21-478f-af8b-ede8d76576f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:25.888878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafc063-0b21-478f-af8b-ede8d76576f8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:25.890499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-af49897f-32a6-4f7b-85eb-87b9ff5c1209 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:25.892792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-af49897f-32a6-4f7b-85eb-87b9ff5c1209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:25.902019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ebafc063-0b21-478f-af8b-ede8d76576f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:25.912315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:25.913511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:25.915092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:25.915092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:25.915092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:25.915092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:25.920946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 231/474] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:25.922730 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafc063-0b21-478f-af8b-ede8d76576f8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:26.040059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 244/475] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:26.048547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafc063-0b21-478f-af8b-ede8d76576f8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:26.088721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faf466-6054-4ae6-9063-0613e70e8246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:26.089553 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faf466-6054-4ae6-9063-0613e70e8246 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:26.089695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49897f-32a6-4f7b-85eb-87b9ff5c1209 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:26.090036 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faf466-6054-4ae6-9063-0613e70e8246 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:26.090417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faf466-6054-4ae6-9063-0613e70e8246 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:26.091260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faf466-6054-4ae6-9063-0613e70e8246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:26.091662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faf466-6054-4ae6-9063-0613e70e8246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:26.093290 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faf466-6054-4ae6-9063-0613e70e8246 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:26.113522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 232/476] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:26.121687 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faf466-6054-4ae6-9063-0613e70e8246 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:26.148427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:26.198142 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:26.248390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faf466-6054-4ae6-9063-0613e70e8246 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:26.291237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb135f-30d6-4c97-a1f5-6e2b7d3b2101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:26.291945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb135f-30d6-4c97-a1f5-6e2b7d3b2101 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:26.292226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb135f-30d6-4c97-a1f5-6e2b7d3b2101 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:26.292498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb135f-30d6-4c97-a1f5-6e2b7d3b2101 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:26.293246 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb135f-30d6-4c97-a1f5-6e2b7d3b2101 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:26.295024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb135f-30d6-4c97-a1f5-6e2b7d3b2101 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:26.315684 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49897f-32a6-4f7b-85eb-87b9ff5c1209 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:26.333696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 233/477] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:50:26 2022] POST /identity/v3/users => generated 249 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:50:26.393155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce75903-3aab-4d36-ba94-2cee1d571ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:26.394478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce75903-3aab-4d36-ba94-2cee1d571ade None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:26.394982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce75903-3aab-4d36-ba94-2cee1d571ade None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:26.395501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce75903-3aab-4d36-ba94-2cee1d571ade None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:26.397066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce75903-3aab-4d36-ba94-2cee1d571ade None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:26.400073 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce75903-3aab-4d36-ba94-2cee1d571ade None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:26.413177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.sql.core [None req-bce75903-3aab-4d36-ba94-2cee1d571ade None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 07:50:26.413177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 04 07:50:26.413177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [parameters: {'id': '446f315819734b2e8084e34f288bf677', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 04 07:50:26.413177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80143) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-bce75903-3aab-4d36-ba94-2cee1d571ade None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application conn.query(q) Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result.read() Jul 04 07:50:26.446528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return ref.to_dict() Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 07:50:26.447721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application next(self.gen) Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application yield resource Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application next(self.gen) Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise self.value Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application session.commit() Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 07:50:26.449005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.session.flush() Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application compat.raise_( Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 07:50:26.450183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 07:50:26.451347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application conn.query(q) Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result.read() Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 07:50:26.452425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [parameters: {'id': '446f315819734b2e8084e34f288bf677', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:26.453444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 04 07:50:26.454469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:26.455712 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 245/478] 10.222.0.43 () {60 vars in 1131 bytes} [Mon Jul 4 07:50:26 2022] POST /identity/v3/roles => generated 144 bytes in 361 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:26.459005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb135f-30d6-4c97-a1f5-6e2b7d3b2101 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:26.705133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb135f-30d6-4c97-a1f5-6e2b7d3b2101 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:26.738324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:26.782638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0649e-8830-4ed4-86f4-7dd42bb0bb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:26.784180 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0649e-8830-4ed4-86f4-7dd42bb0bb14 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:26.784782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0649e-8830-4ed4-86f4-7dd42bb0bb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:26.785353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0649e-8830-4ed4-86f4-7dd42bb0bb14 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:26.808101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0649e-8830-4ed4-86f4-7dd42bb0bb14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-22d0649e-8830-4ed4-86f4-7dd42bb0bb14 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:26.826591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:26.828843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:26.830896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:26.830896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:26.830896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:26.830896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:26.830896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:26.830896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:26.830896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:26.830896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:26.830896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:26.830896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:26.830896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 07:50:26.830896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:26.837672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 234/479] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:50:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:26.839097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce75903-3aab-4d36-ba94-2cee1d571ade None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:27.008929 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:27.036265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce75903-3aab-4d36-ba94-2cee1d571ade None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:27.099274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f7e71-3e8d-43e1-8123-a26a901330f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:27.100411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f7e71-3e8d-43e1-8123-a26a901330f2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:27.100965 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f7e71-3e8d-43e1-8123-a26a901330f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:27.101456 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f7e71-3e8d-43e1-8123-a26a901330f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:27.102906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f7e71-3e8d-43e1-8123-a26a901330f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:27.103453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f7e71-3e8d-43e1-8123-a26a901330f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:27.106079 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f7e71-3e8d-43e1-8123-a26a901330f2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:27.129801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 246/480] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:27.961721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45324b87-1f82-4a11-af18-e1a6d4290920 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:27.962287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45324b87-1f82-4a11-af18-e1a6d4290920 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:27.962652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45324b87-1f82-4a11-af18-e1a6d4290920 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:27.964520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 235/481] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:27.983655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba9aa7-8dc3-4e94-ab92-37de16265f59 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:27.983989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba9aa7-8dc3-4e94-ab92-37de16265f59 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:27.984420 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba9aa7-8dc3-4e94-ab92-37de16265f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:28.014361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b1ba9aa7-8dc3-4e94-ab92-37de16265f59 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:28.015592 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b1ba9aa7-8dc3-4e94-ab92-37de16265f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:28.132303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 247/482] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:28.171022 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45324b87-1f82-4a11-af18-e1a6d4290920 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:28.355579 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45324b87-1f82-4a11-af18-e1a6d4290920 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:28.417214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b6abb-20d4-4b3d-9cfc-363499bbf594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:28.418580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b6abb-20d4-4b3d-9cfc-363499bbf594 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:28.419102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b6abb-20d4-4b3d-9cfc-363499bbf594 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:28.419601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b6abb-20d4-4b3d-9cfc-363499bbf594 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:28.442393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b6abb-20d4-4b3d-9cfc-363499bbf594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-083b6abb-20d4-4b3d-9cfc-363499bbf594 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:28.451226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:28.452201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:28.453026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:28.453026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:28.453026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:28.453026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:28.453026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:28.453026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:28.453026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:28.453026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:28.453026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:28.453026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:28.453026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:28.453026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:28.455587 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 236/483] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:28.462056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba9aa7-8dc3-4e94-ab92-37de16265f59 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:28.606939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:28.641578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba9aa7-8dc3-4e94-ab92-37de16265f59 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:28.700346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f4d5d-9722-4dd6-a8a8-67be1e14037a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:28.701459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f4d5d-9722-4dd6-a8a8-67be1e14037a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:28.701980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f4d5d-9722-4dd6-a8a8-67be1e14037a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:28.702455 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f4d5d-9722-4dd6-a8a8-67be1e14037a None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:28.703467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f4d5d-9722-4dd6-a8a8-67be1e14037a None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:28.704000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f4d5d-9722-4dd6-a8a8-67be1e14037a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:28.705656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f4d5d-9722-4dd6-a8a8-67be1e14037a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:28.722875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 248/484] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:28.728756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b6abb-20d4-4b3d-9cfc-363499bbf594 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:28.777897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:28.813095 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159a1f0-0066-47c0-b50f-de533f698765 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:28.813930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159a1f0-0066-47c0-b50f-de533f698765 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:28.814420 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159a1f0-0066-47c0-b50f-de533f698765 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:28.816082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 249/485] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:50:28.823162 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde0b86-111d-4611-ab8d-d4e62ac4b601 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:28.823495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde0b86-111d-4611-ab8d-d4e62ac4b601 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:28.823726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde0b86-111d-4611-ab8d-d4e62ac4b601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:28.847062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-afde0b86-111d-4611-ab8d-d4e62ac4b601 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:28.848327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-afde0b86-111d-4611-ab8d-d4e62ac4b601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:28.888140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b6abb-20d4-4b3d-9cfc-363499bbf594 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:28.935407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2702-d2e6-4e2c-80a9-93bc4b8e771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:28.937358 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2702-d2e6-4e2c-80a9-93bc4b8e771c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:28.937909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2702-d2e6-4e2c-80a9-93bc4b8e771c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:28.938407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2702-d2e6-4e2c-80a9-93bc4b8e771c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:28.955009 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d2702-d2e6-4e2c-80a9-93bc4b8e771c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:28.957852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d2702-d2e6-4e2c-80a9-93bc4b8e771c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:28.963051 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 250/486] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:50:28.969055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598dcf68-518c-41d8-9ada-3cd5f88f1bce None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:28.969498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598dcf68-518c-41d8-9ada-3cd5f88f1bce None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:28.969679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598dcf68-518c-41d8-9ada-3cd5f88f1bce None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:28.971333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 251/487] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:28.981700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7312595-0ad6-4db3-a143-a10ed661d806 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:28.981700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7312595-0ad6-4db3-a143-a10ed661d806 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:28.981700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7312595-0ad6-4db3-a143-a10ed661d806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ef8d2702-d2e6-4e2c-80a9-93bc4b8e771c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:28.994915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:28.997404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:28.999829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:28.999829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:28.999829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:28.999829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:28.999829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 237/488] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:28.999829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2702-d2e6-4e2c-80a9-93bc4b8e771c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:29.003681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d7312595-0ad6-4db3-a143-a10ed661d806 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:29.005247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d7312595-0ad6-4db3-a143-a10ed661d806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:29.078794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:29.085537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 252/489] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:50:29.087090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7312595-0ad6-4db3-a143-a10ed661d806 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:29.163074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:29.192036 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2702-d2e6-4e2c-80a9-93bc4b8e771c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:29.230639 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7312595-0ad6-4db3-a143-a10ed661d806 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:29.250967 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9eb6dc-cfa7-4181-8b58-97336693afd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:29.252337 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eb6dc-cfa7-4181-8b58-97336693afd7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:29.252803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eb6dc-cfa7-4181-8b58-97336693afd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:29.253223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eb6dc-cfa7-4181-8b58-97336693afd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:29.268835 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9eb6dc-cfa7-4181-8b58-97336693afd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2a9eb6dc-cfa7-4181-8b58-97336693afd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:29.279729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:29.280855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:29.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:29.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:29.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:29.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:29.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:29.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:29.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:29.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:29.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:29.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:29.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:29.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:29.285112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 238/490] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:29.286543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9eb6dc-cfa7-4181-8b58-97336693afd7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:29.293970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8281b-4c48-425b-948c-24ef2f1f6544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:29.295337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8281b-4c48-425b-948c-24ef2f1f6544 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:29.295923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8281b-4c48-425b-948c-24ef2f1f6544 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:29.296610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8281b-4c48-425b-948c-24ef2f1f6544 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:29.297783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8281b-4c48-425b-948c-24ef2f1f6544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:29.298869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8281b-4c48-425b-948c-24ef2f1f6544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:29.301776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8281b-4c48-425b-948c-24ef2f1f6544 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:29.326578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 253/491] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:29.328613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8281b-4c48-425b-948c-24ef2f1f6544 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:29.403785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:29.445116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9eb6dc-cfa7-4181-8b58-97336693afd7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:29.452385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8281b-4c48-425b-948c-24ef2f1f6544 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:29.490619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589250d-3fd1-4878-bedf-18bdea8da58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:29.491891 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd437b4-5b03-465a-973f-1957f55834b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:29.492080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589250d-3fd1-4878-bedf-18bdea8da58f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:29.492439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589250d-3fd1-4878-bedf-18bdea8da58f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:29.493007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589250d-3fd1-4878-bedf-18bdea8da58f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:29.493460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd437b4-5b03-465a-973f-1957f55834b6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:29.493940 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd437b4-5b03-465a-973f-1957f55834b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:29.494360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd437b4-5b03-465a-973f-1957f55834b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:29.495284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd437b4-5b03-465a-973f-1957f55834b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:29.495747 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd437b4-5b03-465a-973f-1957f55834b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:29.497328 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd437b4-5b03-465a-973f-1957f55834b6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:29.507844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 254/492] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:29.509145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd437b4-5b03-465a-973f-1957f55834b6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:29.509359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589250d-3fd1-4878-bedf-18bdea8da58f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2589250d-3fd1-4878-bedf-18bdea8da58f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:29.519620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:29.520816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:29.521800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:29.521800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:29.521800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:29.521800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:29.521800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:29.521800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:29.521800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:29.521800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:29.521800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:29.521800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:29.521800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:29.521800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:29.524917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 239/493] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:29.526244 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589250d-3fd1-4878-bedf-18bdea8da58f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:29.531742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:29.649209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd437b4-5b03-465a-973f-1957f55834b6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:29.671469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589250d-3fd1-4878-bedf-18bdea8da58f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:29.675866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:29.691492 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee223b19-7aac-48f4-9d52-0ea6bdf8cf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:29.692801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee223b19-7aac-48f4-9d52-0ea6bdf8cf56 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:29.693267 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee223b19-7aac-48f4-9d52-0ea6bdf8cf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:29.693549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee223b19-7aac-48f4-9d52-0ea6bdf8cf56 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:29.703546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee223b19-7aac-48f4-9d52-0ea6bdf8cf56 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:29.705065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee223b19-7aac-48f4-9d52-0ea6bdf8cf56 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ee223b19-7aac-48f4-9d52-0ea6bdf8cf56 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:29.724432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:29.727299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:29.728916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:29.728916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:29.728916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:29.728916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:29.728916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:29.728916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:29.728916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:29.728916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:29.728916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:50:29.728916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:29.728916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 255/494] 10.222.0.43 () {56 vars in 1093 bytes} [Mon Jul 4 07:50:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:29.728916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee223b19-7aac-48f4-9d52-0ea6bdf8cf56 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:29.733226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4187f-5dac-4527-bf57-c91a2bd18b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:29.734946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4187f-5dac-4527-bf57-c91a2bd18b34 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:29.736768 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4187f-5dac-4527-bf57-c91a2bd18b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:29.737094 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4187f-5dac-4527-bf57-c91a2bd18b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:29.753319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4187f-5dac-4527-bf57-c91a2bd18b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:29.755205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4187f-5dac-4527-bf57-c91a2bd18b34 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-99e4187f-5dac-4527-bf57-c91a2bd18b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:29.775379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:29.776637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:29.777762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:29.777762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:29.777762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:29.777762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:29.781506 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 240/495] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:29.783081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadf189-cb95-4c75-904e-03b316c2702e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:29.783794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadf189-cb95-4c75-904e-03b316c2702e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:29.783794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadf189-cb95-4c75-904e-03b316c2702e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:29.785356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 241/496] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:29.786020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faadf189-cb95-4c75-904e-03b316c2702e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:29.875646 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:29.929441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faadf189-cb95-4c75-904e-03b316c2702e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:29.942918 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee223b19-7aac-48f4-9d52-0ea6bdf8cf56 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:29.967909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be71573b-70c9-4943-84fe-a89b6dccfeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:29.968775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be71573b-70c9-4943-84fe-a89b6dccfeff None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:29.969153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be71573b-70c9-4943-84fe-a89b6dccfeff None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:29.969522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be71573b-70c9-4943-84fe-a89b6dccfeff None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:29.970524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be71573b-70c9-4943-84fe-a89b6dccfeff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:29.970926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be71573b-70c9-4943-84fe-a89b6dccfeff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:29.972479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be71573b-70c9-4943-84fe-a89b6dccfeff None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:30.010327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f268b8e-aec3-4231-bae4-01a0863869b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:30.011266 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f268b8e-aec3-4231-bae4-01a0863869b7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:30.011713 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f268b8e-aec3-4231-bae4-01a0863869b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:30.012142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f268b8e-aec3-4231-bae4-01a0863869b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:30.013019 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f268b8e-aec3-4231-bae4-01a0863869b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:30.013413 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f268b8e-aec3-4231-bae4-01a0863869b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:30.020367 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 242/497] 10.222.0.43 () {56 vars in 1133 bytes} [Mon Jul 4 07:50:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:30.020367 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be71573b-70c9-4943-84fe-a89b6dccfeff None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:30.021052 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f268b8e-aec3-4231-bae4-01a0863869b7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:30.036763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 256/498] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:30.039130 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc74e7-821a-4a48-bda0-0dbbf30974f2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:30.039524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc74e7-821a-4a48-bda0-0dbbf30974f2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:30.039950 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc74e7-821a-4a48-bda0-0dbbf30974f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:30.060206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-08fc74e7-821a-4a48-bda0-0dbbf30974f2 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:30.061891 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-08fc74e7-821a-4a48-bda0-0dbbf30974f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:30.126950 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:30.169542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be71573b-70c9-4943-84fe-a89b6dccfeff None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:30.181671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 257/499] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:30.181671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc74e7-821a-4a48-bda0-0dbbf30974f2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:30.201763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:30.239889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d8570-10f7-4e5f-96dd-6a275f97035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:30.240793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d8570-10f7-4e5f-96dd-6a275f97035d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:30.241250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d8570-10f7-4e5f-96dd-6a275f97035d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:30.242163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d8570-10f7-4e5f-96dd-6a275f97035d None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:30.243403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d8570-10f7-4e5f-96dd-6a275f97035d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:30.243907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d8570-10f7-4e5f-96dd-6a275f97035d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:30.245830 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d8570-10f7-4e5f-96dd-6a275f97035d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:30.263828 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 243/500] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:30.264720 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d8570-10f7-4e5f-96dd-6a275f97035d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:30.333106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc74e7-821a-4a48-bda0-0dbbf30974f2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:30.372790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ce5b7-e86f-41fb-bcfe-2bb06fd0e183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:30.373449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ce5b7-e86f-41fb-bcfe-2bb06fd0e183 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:30.373772 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ce5b7-e86f-41fb-bcfe-2bb06fd0e183 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:30.374053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ce5b7-e86f-41fb-bcfe-2bb06fd0e183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:30.381501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d8570-10f7-4e5f-96dd-6a275f97035d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:30.418793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8019fb-d829-48e8-bb61-9e278b34f120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:30.419607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8019fb-d829-48e8-bb61-9e278b34f120 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:30.419891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8019fb-d829-48e8-bb61-9e278b34f120 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:30.420166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8019fb-d829-48e8-bb61-9e278b34f120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:30.432472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8019fb-d829-48e8-bb61-9e278b34f120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:30.433994 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8019fb-d829-48e8-bb61-9e278b34f120 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:30.436184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-4b8019fb-d829-48e8-bb61-9e278b34f120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:30.453658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:30.455305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:30.456945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:30.456945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:30.456945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:30.456945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:30.456945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 244/501] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:30.456945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8019fb-d829-48e8-bb61-9e278b34f120 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:30.472307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ce5b7-e86f-41fb-bcfe-2bb06fd0e183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:30.473943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ce5b7-e86f-41fb-bcfe-2bb06fd0e183 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:30.487200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:30.590518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 258/502] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:30.590518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf04d0-b6a9-4afe-a480-5ad5f3ddb804 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:30.590518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf04d0-b6a9-4afe-a480-5ad5f3ddb804 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:30.590518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf04d0-b6a9-4afe-a480-5ad5f3ddb804 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:30.590518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 259/503] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:30.591480 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf04d0-b6a9-4afe-a480-5ad5f3ddb804 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:30.593686 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8019fb-d829-48e8-bb61-9e278b34f120 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:30.651697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c20e8-13d6-4296-9cd4-90d53cf87a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:30.652799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c20e8-13d6-4296-9cd4-90d53cf87a9a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:30.653304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c20e8-13d6-4296-9cd4-90d53cf87a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:30.653846 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c20e8-13d6-4296-9cd4-90d53cf87a9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:30.672204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c20e8-13d6-4296-9cd4-90d53cf87a9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:30.675012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1d4c20e8-13d6-4296-9cd4-90d53cf87a9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:30.687257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:30.688442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:30.689443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:30.689443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:30.689443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:30.689443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:30.689443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:30.689443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:30.689443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:30.689443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:30.689443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:30.689443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:30.689443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:30.689443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:30.690088 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 245/504] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:30.690984 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c20e8-13d6-4296-9cd4-90d53cf87a9a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:30.715215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf04d0-b6a9-4afe-a480-5ad5f3ddb804 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:30.753887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0d5ce-e508-49a6-884f-4c5963d59e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:30.755074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0d5ce-e508-49a6-884f-4c5963d59e30 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:30.755349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0d5ce-e508-49a6-884f-4c5963d59e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:30.755617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0d5ce-e508-49a6-884f-4c5963d59e30 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:30.768119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0d5ce-e508-49a6-884f-4c5963d59e30 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:30.771221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0d5ce-e508-49a6-884f-4c5963d59e30 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7bc0d5ce-e508-49a6-884f-4c5963d59e30 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:30.798406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:30.799596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:30.801001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:30.801001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:30.801001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:30.801001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:30.801001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:30.801001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:30.801001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:30.801001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:30.801001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 07:50:30.801001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:30.801537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 260/505] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:30.802033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:30.802822 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0d5ce-e508-49a6-884f-4c5963d59e30 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:30.827080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c20e8-13d6-4296-9cd4-90d53cf87a9a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:30.866855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992a347-4f2a-4b1b-8a09-f431bb8b8630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:30.867471 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992a347-4f2a-4b1b-8a09-f431bb8b8630 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:30.867751 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992a347-4f2a-4b1b-8a09-f431bb8b8630 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:30.868025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992a347-4f2a-4b1b-8a09-f431bb8b8630 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:30.868763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992a347-4f2a-4b1b-8a09-f431bb8b8630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:30.869072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992a347-4f2a-4b1b-8a09-f431bb8b8630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:30.870547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992a347-4f2a-4b1b-8a09-f431bb8b8630 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:30.885453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 246/506] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:30.887251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624f09c-72ef-486f-9000-2429e5452dab None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:30.887628 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624f09c-72ef-486f-9000-2429e5452dab None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:30.887985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624f09c-72ef-486f-9000-2429e5452dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:30.908645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3624f09c-72ef-486f-9000-2429e5452dab None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:30.909955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3624f09c-72ef-486f-9000-2429e5452dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:30.945221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0d5ce-e508-49a6-884f-4c5963d59e30 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:30.982702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285a8d8-038c-4ad5-81db-477c9fd86ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:30.983510 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285a8d8-038c-4ad5-81db-477c9fd86ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:30.983793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285a8d8-038c-4ad5-81db-477c9fd86ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:30.984066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285a8d8-038c-4ad5-81db-477c9fd86ee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:30.986192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:30.994461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 247/507] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:30.995330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624f09c-72ef-486f-9000-2429e5452dab None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:30.995748 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285a8d8-038c-4ad5-81db-477c9fd86ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:30.997333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285a8d8-038c-4ad5-81db-477c9fd86ee4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f285a8d8-038c-4ad5-81db-477c9fd86ee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:31.018681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:31.020490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:31.022368 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:31.022368 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:31.022368 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:31.022368 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:31.023057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 261/508] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:31.024070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285a8d8-038c-4ad5-81db-477c9fd86ee4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:31.114421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:31.148502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624f09c-72ef-486f-9000-2429e5452dab None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:31.154145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285a8d8-038c-4ad5-81db-477c9fd86ee4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:31.188106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae60aac-0a7b-4649-b6a9-da48fb5a3c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:31.188807 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae60aac-0a7b-4649-b6a9-da48fb5a3c44 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:31.189100 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae60aac-0a7b-4649-b6a9-da48fb5a3c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:31.189393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae60aac-0a7b-4649-b6a9-da48fb5a3c44 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:31.190190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae60aac-0a7b-4649-b6a9-da48fb5a3c44 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:31.190499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae60aac-0a7b-4649-b6a9-da48fb5a3c44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:31.191923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae60aac-0a7b-4649-b6a9-da48fb5a3c44 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:31.199503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89a868-494e-4e27-843b-e1c8f5d24dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:31.200201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89a868-494e-4e27-843b-e1c8f5d24dbc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:31.200480 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89a868-494e-4e27-843b-e1c8f5d24dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:31.200750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89a868-494e-4e27-843b-e1c8f5d24dbc None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:31.201564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 248/509] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:31.201807 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89a868-494e-4e27-843b-e1c8f5d24dbc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:31.202050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89a868-494e-4e27-843b-e1c8f5d24dbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:31.202536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae60aac-0a7b-4649-b6a9-da48fb5a3c44 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:31.203658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89a868-494e-4e27-843b-e1c8f5d24dbc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:31.246829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 262/510] 10.222.0.43 () {56 vars in 1137 bytes} [Mon Jul 4 07:50:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:31.247752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89a868-494e-4e27-843b-e1c8f5d24dbc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:31.300683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:31.323359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae60aac-0a7b-4649-b6a9-da48fb5a3c44 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:31.364242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce35ec-17f1-492a-b5d7-1698fae52f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:31.365061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce35ec-17f1-492a-b5d7-1698fae52f6d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:31.365340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce35ec-17f1-492a-b5d7-1698fae52f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:31.365608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce35ec-17f1-492a-b5d7-1698fae52f6d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:31.369513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:31.370446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89a868-494e-4e27-843b-e1c8f5d24dbc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:31.379582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce35ec-17f1-492a-b5d7-1698fae52f6d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:31.381637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce35ec-17f1-492a-b5d7-1698fae52f6d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-6fce35ec-17f1-492a-b5d7-1698fae52f6d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:31.400826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:31.402649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:31.403912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:31.403912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:31.403912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:31.403912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:31.403912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:31.403912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:31.403912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:31.403912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:31.403912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 07:50:31.403912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:31.403912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 249/511] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:31.403912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce35ec-17f1-492a-b5d7-1698fae52f6d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:31.420120 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d9e2e-850c-42e5-8936-51f7ee8e6b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:31.420928 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d9e2e-850c-42e5-8936-51f7ee8e6b29 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:31.421270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d9e2e-850c-42e5-8936-51f7ee8e6b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:31.421563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d9e2e-850c-42e5-8936-51f7ee8e6b29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:31.433082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d9e2e-850c-42e5-8936-51f7ee8e6b29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:31.434838 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d9e2e-850c-42e5-8936-51f7ee8e6b29 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-819d9e2e-850c-42e5-8936-51f7ee8e6b29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:31.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:31.456618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:31.457528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:31.457528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:31.457528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:31.457528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:31.457528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 263/512] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:31.457528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d9e2e-850c-42e5-8936-51f7ee8e6b29 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:31.531309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce35ec-17f1-492a-b5d7-1698fae52f6d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:31.568959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:31.571592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4c4c7-3b6e-4262-8422-1c9563586e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:31.572632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4c4c7-3b6e-4262-8422-1c9563586e2b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:31.573105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4c4c7-3b6e-4262-8422-1c9563586e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:31.573560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4c4c7-3b6e-4262-8422-1c9563586e2b None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:31.574715 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4c4c7-3b6e-4262-8422-1c9563586e2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:31.575234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4c4c7-3b6e-4262-8422-1c9563586e2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:31.577596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4c4c7-3b6e-4262-8422-1c9563586e2b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:31.579060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d9e2e-850c-42e5-8936-51f7ee8e6b29 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:31.595898 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 250/513] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:31.597062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4c4c7-3b6e-4262-8422-1c9563586e2b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:31.625313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763f50f-d617-44b1-a84a-4d76ab68821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:31.626184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763f50f-d617-44b1-a84a-4d76ab68821a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:31.626454 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763f50f-d617-44b1-a84a-4d76ab68821a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:31.626731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763f50f-d617-44b1-a84a-4d76ab68821a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:31.639029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763f50f-d617-44b1-a84a-4d76ab68821a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:31.640516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763f50f-d617-44b1-a84a-4d76ab68821a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d763f50f-d617-44b1-a84a-4d76ab68821a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:31.660415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:31.661351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:31.669141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:31.669141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:31.669141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:31.669141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:31.669141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 264/514] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:31.669141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763f50f-d617-44b1-a84a-4d76ab68821a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:31.669485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:31.711393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4c4c7-3b6e-4262-8422-1c9563586e2b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:31.747718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40a56f-7d7b-4cb7-9f47-933f044597d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:31.748378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40a56f-7d7b-4cb7-9f47-933f044597d6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:31.748656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40a56f-7d7b-4cb7-9f47-933f044597d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:31.748936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40a56f-7d7b-4cb7-9f47-933f044597d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:31.819701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763f50f-d617-44b1-a84a-4d76ab68821a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:31.831684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40a56f-7d7b-4cb7-9f47-933f044597d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:31.833157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40a56f-7d7b-4cb7-9f47-933f044597d6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:31.863340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c5f00-1e3c-45b5-a443-2a85211d53d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:31.864409 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c5f00-1e3c-45b5-a443-2a85211d53d1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:31.864866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c5f00-1e3c-45b5-a443-2a85211d53d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:31.865314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c5f00-1e3c-45b5-a443-2a85211d53d1 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:31.866758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c5f00-1e3c-45b5-a443-2a85211d53d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:31.867165 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c5f00-1e3c-45b5-a443-2a85211d53d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:31.868819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c5f00-1e3c-45b5-a443-2a85211d53d1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:31.889813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 265/515] 10.222.0.43 () {56 vars in 1137 bytes} [Mon Jul 4 07:50:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:31.890526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c5f00-1e3c-45b5-a443-2a85211d53d1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:31.900381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:31.909496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:31.965893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 251/516] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:31.966595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40a56f-7d7b-4cb7-9f47-933f044597d6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:32.009614 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c5f00-1e3c-45b5-a443-2a85211d53d1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:32.048781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2605d37-4e45-4da1-92c0-c27f65205e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:32.049471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2605d37-4e45-4da1-92c0-c27f65205e35 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:32.049796 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2605d37-4e45-4da1-92c0-c27f65205e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:32.050071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2605d37-4e45-4da1-92c0-c27f65205e35 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:32.050826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2605d37-4e45-4da1-92c0-c27f65205e35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:32.051137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2605d37-4e45-4da1-92c0-c27f65205e35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:32.052577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2605d37-4e45-4da1-92c0-c27f65205e35 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:32.065778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 266/517] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:32.066638 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2605d37-4e45-4da1-92c0-c27f65205e35 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:32.089989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40a56f-7d7b-4cb7-9f47-933f044597d6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:32.142283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72fa23-9896-4635-bda3-96701644ee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:32.143472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72fa23-9896-4635-bda3-96701644ee10 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:32.143928 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72fa23-9896-4635-bda3-96701644ee10 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:32.144423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72fa23-9896-4635-bda3-96701644ee10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:32.150633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:32.155724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72fa23-9896-4635-bda3-96701644ee10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:32.158235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72fa23-9896-4635-bda3-96701644ee10 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-be72fa23-9896-4635-bda3-96701644ee10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:32.182533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:32.184021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:32.185315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:32.185315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:32.185315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:32.185315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:32.185315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:32.185315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:32.185315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:32.185315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:32.189828 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 252/518] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:32.191030 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72fa23-9896-4635-bda3-96701644ee10 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:32.205884 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2605d37-4e45-4da1-92c0-c27f65205e35 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:32.215387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:32.245242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2845f08-e0ce-48f6-87d2-48ffc8eed69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:32.245881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2845f08-e0ce-48f6-87d2-48ffc8eed69c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:32.246148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2845f08-e0ce-48f6-87d2-48ffc8eed69c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:32.246435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2845f08-e0ce-48f6-87d2-48ffc8eed69c None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:32.247157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2845f08-e0ce-48f6-87d2-48ffc8eed69c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:32.247462 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2845f08-e0ce-48f6-87d2-48ffc8eed69c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:32.248944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2845f08-e0ce-48f6-87d2-48ffc8eed69c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:32.264956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 267/519] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:32.265843 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2845f08-e0ce-48f6-87d2-48ffc8eed69c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:32.343952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72fa23-9896-4635-bda3-96701644ee10 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:32.385654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c561d6c-4612-4826-b03e-c04b2c2db705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:32.386326 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c561d6c-4612-4826-b03e-c04b2c2db705 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:32.386611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c561d6c-4612-4826-b03e-c04b2c2db705 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:32.386891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c561d6c-4612-4826-b03e-c04b2c2db705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:32.391127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2845f08-e0ce-48f6-87d2-48ffc8eed69c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:32.426904 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:32.440269 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a5125-5078-4b39-b54d-7a4f040740de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:32.441259 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a5125-5078-4b39-b54d-7a4f040740de None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:32.441603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a5125-5078-4b39-b54d-7a4f040740de None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:32.441988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a5125-5078-4b39-b54d-7a4f040740de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:32.454868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a5125-5078-4b39-b54d-7a4f040740de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:32.456384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a5125-5078-4b39-b54d-7a4f040740de None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:32.474487 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c561d6c-4612-4826-b03e-c04b2c2db705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b47a5125-5078-4b39-b54d-7a4f040740de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:32.475400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:32.476598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:32.479198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:32.479198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:32.479198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:32.479198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:32.479695 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c561d6c-4612-4826-b03e-c04b2c2db705 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:32.479872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 268/520] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:32.479872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a5125-5078-4b39-b54d-7a4f040740de None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:32.503502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:32.607158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 253/521] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:32.608057 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c561d6c-4612-4826-b03e-c04b2c2db705 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:32.611916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a5125-5078-4b39-b54d-7a4f040740de None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:32.655567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a426f-63a8-4415-8f8f-5e71076deea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:32.656649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a426f-63a8-4415-8f8f-5e71076deea0 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:32.657112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a426f-63a8-4415-8f8f-5e71076deea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:32.657559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a426f-63a8-4415-8f8f-5e71076deea0 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:32.658352 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a426f-63a8-4415-8f8f-5e71076deea0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:32.659837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a426f-63a8-4415-8f8f-5e71076deea0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:32.683346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 269/522] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:50:32 2022] POST /identity/v3/users => generated 247 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:50:32.684403 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a426f-63a8-4415-8f8f-5e71076deea0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:32.739604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:32.776640 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c561d6c-4612-4826-b03e-c04b2c2db705 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:32.803672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:32.837081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a34f4f-baf4-4ab2-9d79-eb0578102d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:32.838160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a34f4f-baf4-4ab2-9d79-eb0578102d3b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:32.839009 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a34f4f-baf4-4ab2-9d79-eb0578102d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:32.839009 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a34f4f-baf4-4ab2-9d79-eb0578102d3b None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:32.840228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a34f4f-baf4-4ab2-9d79-eb0578102d3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:32.840762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a34f4f-baf4-4ab2-9d79-eb0578102d3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:32.843251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a34f4f-baf4-4ab2-9d79-eb0578102d3b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:32.851798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a426f-63a8-4415-8f8f-5e71076deea0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:32.870433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 254/523] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:32.871805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a34f4f-baf4-4ab2-9d79-eb0578102d3b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:32.916757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6a1a7-fef1-48c8-9617-ebe120bfed2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:32.918805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6a1a7-fef1-48c8-9617-ebe120bfed2d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:32.920681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6a1a7-fef1-48c8-9617-ebe120bfed2d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:32.921188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6a1a7-fef1-48c8-9617-ebe120bfed2d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:32.932494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6a1a7-fef1-48c8-9617-ebe120bfed2d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:32.934286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6a1a7-fef1-48c8-9617-ebe120bfed2d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b8c6a1a7-fef1-48c8-9617-ebe120bfed2d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:32.953792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:32.955104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:32.957794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:32.957794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:32.957794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:32.957794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:32.957794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:32.957794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:32.957794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:32.957794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:32.957794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 07:50:32.957794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:32.957794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 270/524] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:32.957794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6a1a7-fef1-48c8-9617-ebe120bfed2d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:33.070744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a34f4f-baf4-4ab2-9d79-eb0578102d3b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:33.132907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:33.141274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6a1a7-fef1-48c8-9617-ebe120bfed2d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:33.141529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d46a1-f08c-46ba-9e03-80137539a996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:33.142168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d46a1-f08c-46ba-9e03-80137539a996 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:33.142449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d46a1-f08c-46ba-9e03-80137539a996 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:33.143278 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d46a1-f08c-46ba-9e03-80137539a996 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:33.143693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d46a1-f08c-46ba-9e03-80137539a996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:33.144937 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d46a1-f08c-46ba-9e03-80137539a996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:33.146418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d46a1-f08c-46ba-9e03-80137539a996 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:33.167561 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 255/525] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:33.169103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d46a1-f08c-46ba-9e03-80137539a996 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:33.200890 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c971b68-d778-4fea-85be-9985de309a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:33.202438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c971b68-d778-4fea-85be-9985de309a16 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:33.202930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c971b68-d778-4fea-85be-9985de309a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:33.203526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c971b68-d778-4fea-85be-9985de309a16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:33.218117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c971b68-d778-4fea-85be-9985de309a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:33.219673 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c971b68-d778-4fea-85be-9985de309a16 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:33.222076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6c971b68-d778-4fea-85be-9985de309a16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:33.239738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:33.240784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:33.242350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:33.242350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:33.242350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:33.242350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:33.242350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 271/526] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:33.242350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c971b68-d778-4fea-85be-9985de309a16 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:33.315372 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d46a1-f08c-46ba-9e03-80137539a996 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:33.355015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8c8c2-6edd-4b08-9098-59efdaa26e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:33.356157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8c8c2-6edd-4b08-9098-59efdaa26e68 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:33.356726 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8c8c2-6edd-4b08-9098-59efdaa26e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:33.357213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8c8c2-6edd-4b08-9098-59efdaa26e68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:33.358313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8c8c2-6edd-4b08-9098-59efdaa26e68 None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=15211e42ec9d4c1699d877f0b55ce918) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:33.358854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8c8c2-6edd-4b08-9098-59efdaa26e68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:33.360598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8c8c2-6edd-4b08-9098-59efdaa26e68 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:33.364540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c971b68-d778-4fea-85be-9985de309a16 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:33.367797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 256/527] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:33 2022] GET /identity/v3/role_assignments?user.id=15211e42ec9d4c1699d877f0b55ce918&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:33.368989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8c8c2-6edd-4b08-9098-59efdaa26e68 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:33.417868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f177d3-c358-4a74-b14e-109a392c2470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:33.418562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f177d3-c358-4a74-b14e-109a392c2470 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:33.418811 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f177d3-c358-4a74-b14e-109a392c2470 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:33.419194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f177d3-c358-4a74-b14e-109a392c2470 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:33.420079 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f177d3-c358-4a74-b14e-109a392c2470 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:33.420389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f177d3-c358-4a74-b14e-109a392c2470 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:33.421960 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f177d3-c358-4a74-b14e-109a392c2470 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:33.445121 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 272/528] 10.222.0.43 () {56 vars in 1139 bytes} [Mon Jul 4 07:50:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:33.453236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f177d3-c358-4a74-b14e-109a392c2470 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:33.458195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:33.531486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:33.534755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8c8c2-6edd-4b08-9098-59efdaa26e68 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:33.593283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb661a28-15a9-47d6-a3c2-91029497f4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:33.594704 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb661a28-15a9-47d6-a3c2-91029497f4fa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:33.595184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb661a28-15a9-47d6-a3c2-91029497f4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:33.596035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb661a28-15a9-47d6-a3c2-91029497f4fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:33.604668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb661a28-15a9-47d6-a3c2-91029497f4fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:33.606312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb661a28-15a9-47d6-a3c2-91029497f4fa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:33.611959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f177d3-c358-4a74-b14e-109a392c2470 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-eb661a28-15a9-47d6-a3c2-91029497f4fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:33.623741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:33.624965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:33.625995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:33.625995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:33.625995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:33.625995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:33.625995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:33.625995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:33.625995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:33.625995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:33.633388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 257/529] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:33.634768 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb661a28-15a9-47d6-a3c2-91029497f4fa None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:33.659595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d673db7-6916-4548-9c07-5402c1f24f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:33.660446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d673db7-6916-4548-9c07-5402c1f24f4c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:33.660899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d673db7-6916-4548-9c07-5402c1f24f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:33.661262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d673db7-6916-4548-9c07-5402c1f24f4c None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:33.662057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d673db7-6916-4548-9c07-5402c1f24f4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:33.662527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d673db7-6916-4548-9c07-5402c1f24f4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:33.664635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d673db7-6916-4548-9c07-5402c1f24f4c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:33.683908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 273/530] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:33.683908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d673db7-6916-4548-9c07-5402c1f24f4c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:33.765670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:33.781458 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb661a28-15a9-47d6-a3c2-91029497f4fa None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:33.812566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d673db7-6916-4548-9c07-5402c1f24f4c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:33.820521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b118a5f-8e10-4f99-82e6-9a45f310e4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:33.821070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b118a5f-8e10-4f99-82e6-9a45f310e4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:33.821431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b118a5f-8e10-4f99-82e6-9a45f310e4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:33.821830 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b118a5f-8e10-4f99-82e6-9a45f310e4c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:33.853896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:33.855563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f3527-2c33-4630-a715-e9bfdf63da2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:33.856673 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f3527-2c33-4630-a715-e9bfdf63da2b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:33.857413 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f3527-2c33-4630-a715-e9bfdf63da2b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:33.857928 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f3527-2c33-4630-a715-e9bfdf63da2b None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:33.859545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f3527-2c33-4630-a715-e9bfdf63da2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:33.860098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f3527-2c33-4630-a715-e9bfdf63da2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:33.862398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f3527-2c33-4630-a715-e9bfdf63da2b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:33.886877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 274/531] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:33.888390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f3527-2c33-4630-a715-e9bfdf63da2b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:33.903549 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b118a5f-8e10-4f99-82e6-9a45f310e4c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:33.905047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b118a5f-8e10-4f99-82e6-9a45f310e4c7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:34.043719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f3527-2c33-4630-a715-e9bfdf63da2b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:34.049151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 258/532] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:34.050182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b118a5f-8e10-4f99-82e6-9a45f310e4c7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:34.083436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:34.097270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec845e-c369-4764-9b71-2dc6a0af8dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:34.098384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec845e-c369-4764-9b71-2dc6a0af8dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:34.098674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec845e-c369-4764-9b71-2dc6a0af8dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:34.098955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec845e-c369-4764-9b71-2dc6a0af8dc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:34.111967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec845e-c369-4764-9b71-2dc6a0af8dc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:34.113743 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec845e-c369-4764-9b71-2dc6a0af8dc7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-85ec845e-c369-4764-9b71-2dc6a0af8dc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:34.134846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:34.135769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:34.136655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:34.136655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:34.136655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:34.136655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:34.136655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:34.136655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:34.136655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:34.136655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:34.143661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 275/533] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:34.145057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec845e-c369-4764-9b71-2dc6a0af8dc7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:34.146678 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:34.204607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b118a5f-8e10-4f99-82e6-9a45f310e4c7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:34.251473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fcb4a-267b-4aaa-a426-58ce0b58ac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:34.252172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fcb4a-267b-4aaa-a426-58ce0b58ac30 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:34.252459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fcb4a-267b-4aaa-a426-58ce0b58ac30 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:34.252739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fcb4a-267b-4aaa-a426-58ce0b58ac30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:34.253589 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737fcb4a-267b-4aaa-a426-58ce0b58ac30 None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=061ffcf242ff4155b92daa8c3888fd10) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:34.253947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737fcb4a-267b-4aaa-a426-58ce0b58ac30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:34.255739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737fcb4a-267b-4aaa-a426-58ce0b58ac30 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:34.271913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 259/534] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:34 2022] GET /identity/v3/role_assignments?user.id=061ffcf242ff4155b92daa8c3888fd10&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:34.275884 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fcb4a-267b-4aaa-a426-58ce0b58ac30 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:34.367386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec845e-c369-4764-9b71-2dc6a0af8dc7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:34.457917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f77e3-fc4a-4352-94e9-7cfe3d124b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:34.458741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f77e3-fc4a-4352-94e9-7cfe3d124b8f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:34.459032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f77e3-fc4a-4352-94e9-7cfe3d124b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:34.459315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f77e3-fc4a-4352-94e9-7cfe3d124b8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:34.475348 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f77e3-fc4a-4352-94e9-7cfe3d124b8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:34.476786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f77e3-fc4a-4352-94e9-7cfe3d124b8f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-837f77e3-fc4a-4352-94e9-7cfe3d124b8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:34.496459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:34.497389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:34.498741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:34.498741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:34.498741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:34.498741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:34.498741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 276/535] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:34.504355 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f77e3-fc4a-4352-94e9-7cfe3d124b8f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:34.519981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fcb4a-267b-4aaa-a426-58ce0b58ac30 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:34.549547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:34.553350 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:34.569817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6047485-36e8-4593-871c-0e6b4505575c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:34.571936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6047485-36e8-4593-871c-0e6b4505575c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:34.572582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6047485-36e8-4593-871c-0e6b4505575c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:34.576888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6047485-36e8-4593-871c-0e6b4505575c None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:34.578895 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6047485-36e8-4593-871c-0e6b4505575c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:34.580368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6047485-36e8-4593-871c-0e6b4505575c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:34.582334 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6047485-36e8-4593-871c-0e6b4505575c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:34.603218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 260/536] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:34.606597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6047485-36e8-4593-871c-0e6b4505575c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:34.656062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f77e3-fc4a-4352-94e9-7cfe3d124b8f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:34.696216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ef22a-a600-43d0-85d6-9eaa8e9cb13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:34.696905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ef22a-a600-43d0-85d6-9eaa8e9cb13f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:34.697194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ef22a-a600-43d0-85d6-9eaa8e9cb13f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:34.697475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ef22a-a600-43d0-85d6-9eaa8e9cb13f None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:34.698256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ef22a-a600-43d0-85d6-9eaa8e9cb13f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:34.698568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ef22a-a600-43d0-85d6-9eaa8e9cb13f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:34.701051 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ef22a-a600-43d0-85d6-9eaa8e9cb13f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:34.717156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 277/537] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:34.722587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ef22a-a600-43d0-85d6-9eaa8e9cb13f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:34.844715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:34.854471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ef22a-a600-43d0-85d6-9eaa8e9cb13f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:34.883758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6047485-36e8-4593-871c-0e6b4505575c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:34.907171 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3b5e2-70f8-4bbb-b46a-16ddd96fe058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:34.908449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3b5e2-70f8-4bbb-b46a-16ddd96fe058 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:34.908969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3b5e2-70f8-4bbb-b46a-16ddd96fe058 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:34.909477 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3b5e2-70f8-4bbb-b46a-16ddd96fe058 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:34.920158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3b5e2-70f8-4bbb-b46a-16ddd96fe058 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:34.921783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3b5e2-70f8-4bbb-b46a-16ddd96fe058 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-65c3b5e2-70f8-4bbb-b46a-16ddd96fe058 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:34.939127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:34.940046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:34.941320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:34.941320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:34.941320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:34.941320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:34.941320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:34.941320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:34.941320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:34.941320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:34.941320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 278/538] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:34.947343 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445c774-0d94-4cda-8903-31e42cf6f11b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:34.947727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445c774-0d94-4cda-8903-31e42cf6f11b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:34.947872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445c774-0d94-4cda-8903-31e42cf6f11b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:34.949827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 279/539] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:34.950541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445c774-0d94-4cda-8903-31e42cf6f11b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:34.967797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcb6c9-37c3-4583-a1d8-af73277913df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:34.969901 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcb6c9-37c3-4583-a1d8-af73277913df None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:34.970409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcb6c9-37c3-4583-a1d8-af73277913df None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:34.971102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcb6c9-37c3-4583-a1d8-af73277913df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:34.973213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcb6c9-37c3-4583-a1d8-af73277913df None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=4a122422a45446c9a63d67b4f931fa47) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:34.973828 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcb6c9-37c3-4583-a1d8-af73277913df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:34.977250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcb6c9-37c3-4583-a1d8-af73277913df None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:34.992322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 261/540] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:34 2022] GET /identity/v3/role_assignments?user.id=4a122422a45446c9a63d67b4f931fa47&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 259 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:34.995153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886f891-edec-4632-b1ea-08129f84c2b7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:34.995661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886f891-edec-4632-b1ea-08129f84c2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:34.997718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886f891-edec-4632-b1ea-08129f84c2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:35.029151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6886f891-edec-4632-b1ea-08129f84c2b7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:35.030886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6886f891-edec-4632-b1ea-08129f84c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:35.108506 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:35.124136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445c774-0d94-4cda-8903-31e42cf6f11b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:35.179871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c54ebe-f37a-4fdd-91aa-1e34751d35d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:35.180488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c54ebe-f37a-4fdd-91aa-1e34751d35d3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:35.180777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c54ebe-f37a-4fdd-91aa-1e34751d35d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:35.181055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c54ebe-f37a-4fdd-91aa-1e34751d35d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:35.181928 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c54ebe-f37a-4fdd-91aa-1e34751d35d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:35.182339 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c54ebe-f37a-4fdd-91aa-1e34751d35d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:35.183911 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c54ebe-f37a-4fdd-91aa-1e34751d35d3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:35.187932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 262/541] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:35.208926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 280/542] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:35.215116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886f891-edec-4632-b1ea-08129f84c2b7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:35.225256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c54ebe-f37a-4fdd-91aa-1e34751d35d3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:35.245701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:35.380457 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c54ebe-f37a-4fdd-91aa-1e34751d35d3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:35.405613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886f891-edec-4632-b1ea-08129f84c2b7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:35.427054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c73973-d807-45a1-9e28-1345387ea9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:35.427894 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c73973-d807-45a1-9e28-1345387ea9fb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:35.428179 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c73973-d807-45a1-9e28-1345387ea9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:35.428461 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c73973-d807-45a1-9e28-1345387ea9fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:35.455392 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c73973-d807-45a1-9e28-1345387ea9fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:35.456280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e2be6-4e3f-4b38-bfa6-e15381e11eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:35.457151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e2be6-4e3f-4b38-bfa6-e15381e11eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:35.457611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e2be6-4e3f-4b38-bfa6-e15381e11eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:35.458087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e2be6-4e3f-4b38-bfa6-e15381e11eb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:35.459563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e2be6-4e3f-4b38-bfa6-e15381e11eb9 None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=429e173890b84ee289abc9793794ddb9) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:35.460158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e2be6-4e3f-4b38-bfa6-e15381e11eb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:35.462126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e2be6-4e3f-4b38-bfa6-e15381e11eb9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:35.469182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 263/543] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:35 2022] GET /identity/v3/role_assignments?user.id=429e173890b84ee289abc9793794ddb9&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-66c73973-d807-45a1-9e28-1345387ea9fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:35.475568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:35.478400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:35.480289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 281/544] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:35.482897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e2be6-4e3f-4b38-bfa6-e15381e11eb9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:35.559166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:35.645781 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e2be6-4e3f-4b38-bfa6-e15381e11eb9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:35.691151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6d09f-55e7-4b70-afbf-55506b732924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:35.692235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6d09f-55e7-4b70-afbf-55506b732924 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:35.692691 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6d09f-55e7-4b70-afbf-55506b732924 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:35.693137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6d09f-55e7-4b70-afbf-55506b732924 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:35.694553 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6d09f-55e7-4b70-afbf-55506b732924 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:35.695066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6d09f-55e7-4b70-afbf-55506b732924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:35.697083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6d09f-55e7-4b70-afbf-55506b732924 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:35.707662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 264/545] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:35.714802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c73973-d807-45a1-9e28-1345387ea9fb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:35.843768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:35.880967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c73973-d807-45a1-9e28-1345387ea9fb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:35.896251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a7a28-e0ee-45d2-a9b8-a210a9696dde None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:35.896832 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a7a28-e0ee-45d2-a9b8-a210a9696dde None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:35.897356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a7a28-e0ee-45d2-a9b8-a210a9696dde None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:35.899965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 265/546] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:50:35.912655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9798-4fcf-4e5b-abc7-fb9427621fa2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:35.912950 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9798-4fcf-4e5b-abc7-fb9427621fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:35.913453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9798-4fcf-4e5b-abc7-fb9427621fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:35.935953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d77e9798-4fcf-4e5b-abc7-fb9427621fa2 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:35.937552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d77e9798-4fcf-4e5b-abc7-fb9427621fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:35.949779 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779c92b-e8a9-4d3c-90e0-580f3a51c74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:35.950610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779c92b-e8a9-4d3c-90e0-580f3a51c74c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:35.950894 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779c92b-e8a9-4d3c-90e0-580f3a51c74c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:35.951170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779c92b-e8a9-4d3c-90e0-580f3a51c74c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:35.979194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779c92b-e8a9-4d3c-90e0-580f3a51c74c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:35.981449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779c92b-e8a9-4d3c-90e0-580f3a51c74c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0779c92b-e8a9-4d3c-90e0-580f3a51c74c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:36.012375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:36.013326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:36.015118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:36.015118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:36.015118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:36.015118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:36.022124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 282/547] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:36.028910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779c92b-e8a9-4d3c-90e0-580f3a51c74c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:36.029447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 266/548] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:36.049980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e9798-4fcf-4e5b-abc7-fb9427621fa2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:36.146165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:36.286673 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779c92b-e8a9-4d3c-90e0-580f3a51c74c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:36.293137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e9798-4fcf-4e5b-abc7-fb9427621fa2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:36.357545 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83441908-346a-4c38-a858-3acda47e34f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:36.358905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83441908-346a-4c38-a858-3acda47e34f2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:36.359438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83441908-346a-4c38-a858-3acda47e34f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:36.359895 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83441908-346a-4c38-a858-3acda47e34f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:36.380710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83441908-346a-4c38-a858-3acda47e34f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:36.394141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-83441908-346a-4c38-a858-3acda47e34f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:36.397859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:36.400416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:36.401941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 267/549] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:36.408752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83441908-346a-4c38-a858-3acda47e34f2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:36.410170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d474deb-8bc6-42e7-8fd1-7b53c9b87c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:36.411645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d474deb-8bc6-42e7-8fd1-7b53c9b87c8c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:36.413025 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d474deb-8bc6-42e7-8fd1-7b53c9b87c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:36.414448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d474deb-8bc6-42e7-8fd1-7b53c9b87c8c None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:36.418283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d474deb-8bc6-42e7-8fd1-7b53c9b87c8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:36.419142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d474deb-8bc6-42e7-8fd1-7b53c9b87c8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:36.423198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d474deb-8bc6-42e7-8fd1-7b53c9b87c8c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:36.449102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 283/550] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:36.455655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d474deb-8bc6-42e7-8fd1-7b53c9b87c8c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:36.597881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:36.608279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83441908-346a-4c38-a858-3acda47e34f2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:36.644620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6eeae4-a90b-45db-b41a-096b76eae74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:36.645677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eeae4-a90b-45db-b41a-096b76eae74a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:36.646162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eeae4-a90b-45db-b41a-096b76eae74a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:36.646658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eeae4-a90b-45db-b41a-096b76eae74a None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:36.647655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6eeae4-a90b-45db-b41a-096b76eae74a None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:36.648197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6eeae4-a90b-45db-b41a-096b76eae74a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:36.650146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6eeae4-a90b-45db-b41a-096b76eae74a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:36.652693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d474deb-8bc6-42e7-8fd1-7b53c9b87c8c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:36.665860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 268/551] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:36.667980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6eeae4-a90b-45db-b41a-096b76eae74a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:36.744102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d00ee-20ab-409b-998b-c797f1753a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:36.744927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d00ee-20ab-409b-998b-c797f1753a89 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:36.745214 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d00ee-20ab-409b-998b-c797f1753a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:36.745493 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d00ee-20ab-409b-998b-c797f1753a89 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:36.767040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d00ee-20ab-409b-998b-c797f1753a89 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:36.768591 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d00ee-20ab-409b-998b-c797f1753a89 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:36.783343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6eeae4-a90b-45db-b41a-096b76eae74a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-210d00ee-20ab-409b-998b-c797f1753a89 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:36.810251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:36.811373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:36.812307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:36.812307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:36.812307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:36.812307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:36.812307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:36.812307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:36.812307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:36.812307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:36.812307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 07:50:36.812307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:36.812307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 284/552] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:50:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:36.822615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d00ee-20ab-409b-998b-c797f1753a89 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:36.854190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33b6f8-278f-4134-b040-3d45a995c06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:36.854692 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33b6f8-278f-4134-b040-3d45a995c06b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:36.854968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33b6f8-278f-4134-b040-3d45a995c06b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:36.855238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33b6f8-278f-4134-b040-3d45a995c06b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:36.866932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33b6f8-278f-4134-b040-3d45a995c06b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:36.868386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33b6f8-278f-4134-b040-3d45a995c06b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-3f33b6f8-278f-4134-b040-3d45a995c06b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:36.886513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:36.887476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:36.888514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:36.888514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:36.888514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:36.888514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:36.892890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 269/553] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:36.894305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfabda-dda9-41c3-955a-1c0f60b1db80 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:36.894589 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfabda-dda9-41c3-955a-1c0f60b1db80 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:36.894863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfabda-dda9-41c3-955a-1c0f60b1db80 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:36.896413 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 270/554] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:36.897332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfabda-dda9-41c3-955a-1c0f60b1db80 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:36.906045 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:36.914285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:36.966376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d00ee-20ab-409b-998b-c797f1753a89 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:37.018333 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfabda-dda9-41c3-955a-1c0f60b1db80 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:37.018860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d94925-a87a-4ff0-a4de-6eeb85fa7580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:37.019553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d94925-a87a-4ff0-a4de-6eeb85fa7580 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:37.019833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d94925-a87a-4ff0-a4de-6eeb85fa7580 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:37.020107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d94925-a87a-4ff0-a4de-6eeb85fa7580 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:37.020959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d94925-a87a-4ff0-a4de-6eeb85fa7580 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:37.021293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d94925-a87a-4ff0-a4de-6eeb85fa7580 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:37.025114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d94925-a87a-4ff0-a4de-6eeb85fa7580 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:37.053727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 285/555] 10.222.0.43 () {56 vars in 1135 bytes} [Mon Jul 4 07:50:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:37.055614 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303f01-a409-4750-8400-9925f00de034 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:37.056139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303f01-a409-4750-8400-9925f00de034 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:37.056802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303f01-a409-4750-8400-9925f00de034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:37.076809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-df303f01-a409-4750-8400-9925f00de034 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:37.077494 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c354133-8028-40da-bd87-fbe7baf1469e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:37.078082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-df303f01-a409-4750-8400-9925f00de034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:37.079279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c354133-8028-40da-bd87-fbe7baf1469e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:37.080073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c354133-8028-40da-bd87-fbe7baf1469e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:37.081179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c354133-8028-40da-bd87-fbe7baf1469e None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:37.083485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c354133-8028-40da-bd87-fbe7baf1469e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:37.084046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c354133-8028-40da-bd87-fbe7baf1469e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:37.085968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c354133-8028-40da-bd87-fbe7baf1469e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:37.105124 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 271/556] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:37.106389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c354133-8028-40da-bd87-fbe7baf1469e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:37.190784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 286/557] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:37.192153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df303f01-a409-4750-8400-9925f00de034 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:37.207118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:37.236748 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:37.260419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c354133-8028-40da-bd87-fbe7baf1469e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:37.305808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751a8f9-3004-4fef-ad6f-f9e081fcb318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:37.307885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751a8f9-3004-4fef-ad6f-f9e081fcb318 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:37.308442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751a8f9-3004-4fef-ad6f-f9e081fcb318 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:37.309247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751a8f9-3004-4fef-ad6f-f9e081fcb318 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:37.321428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df303f01-a409-4750-8400-9925f00de034 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:37.366020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f126541-e321-4eda-99f0-b76bdfe462f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:37.366823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f126541-e321-4eda-99f0-b76bdfe462f5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:37.367108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f126541-e321-4eda-99f0-b76bdfe462f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:37.367388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f126541-e321-4eda-99f0-b76bdfe462f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:37.377716 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f126541-e321-4eda-99f0-b76bdfe462f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:37.379983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f126541-e321-4eda-99f0-b76bdfe462f5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0f126541-e321-4eda-99f0-b76bdfe462f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:37.400762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:37.402650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:37.403601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:37.403601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:37.403601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:37.403601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:37.403601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:37.403601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:37.403601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:37.403601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:37.403601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:50:37.403601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:37.405779 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 287/558] 10.222.0.43 () {56 vars in 1093 bytes} [Mon Jul 4 07:50:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:37.406858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f126541-e321-4eda-99f0-b76bdfe462f5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:37.413940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751a8f9-3004-4fef-ad6f-f9e081fcb318 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:37.417759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751a8f9-3004-4fef-ad6f-f9e081fcb318 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:37.521351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:37.532225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:37.540251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f126541-e321-4eda-99f0-b76bdfe462f5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:37.566106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 272/559] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:37.566106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751a8f9-3004-4fef-ad6f-f9e081fcb318 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:37.584943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a993ca-a9a7-41c5-9312-cb735b00f231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:37.586686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a993ca-a9a7-41c5-9312-cb735b00f231 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:37.587341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a993ca-a9a7-41c5-9312-cb735b00f231 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:37.588013 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a993ca-a9a7-41c5-9312-cb735b00f231 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:37.601903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a993ca-a9a7-41c5-9312-cb735b00f231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-25a993ca-a9a7-41c5-9312-cb735b00f231 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:37.612624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:37.613783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:37.614674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 288/560] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:37.615377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c41d0a-93cc-4028-89e4-a4915eb302b2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:37.615949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c41d0a-93cc-4028-89e4-a4915eb302b2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:37.615949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c41d0a-93cc-4028-89e4-a4915eb302b2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:37.617569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 289/561] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:37.618146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c41d0a-93cc-4028-89e4-a4915eb302b2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:37.686733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751a8f9-3004-4fef-ad6f-f9e081fcb318 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:37.735395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075fc86-3248-4ff6-b337-55b6c4dd9f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:37.736078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075fc86-3248-4ff6-b337-55b6c4dd9f12 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:37.736362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075fc86-3248-4ff6-b337-55b6c4dd9f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:37.736635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075fc86-3248-4ff6-b337-55b6c4dd9f12 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:37.737490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075fc86-3248-4ff6-b337-55b6c4dd9f12 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:37.737834 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075fc86-3248-4ff6-b337-55b6c4dd9f12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:37.739601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075fc86-3248-4ff6-b337-55b6c4dd9f12 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:37.758869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 273/562] 10.222.0.43 () {56 vars in 1133 bytes} [Mon Jul 4 07:50:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:37.759092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c41d0a-93cc-4028-89e4-a4915eb302b2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:37.764524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae8792-ee84-4033-9e89-69a32b264b9b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:37.764813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae8792-ee84-4033-9e89-69a32b264b9b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:37.765092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae8792-ee84-4033-9e89-69a32b264b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:37.782741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0cae8792-ee84-4033-9e89-69a32b264b9b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:37.783978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0cae8792-ee84-4033-9e89-69a32b264b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:37.804233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36867d-a538-4752-b76e-8070a3e071df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:37.805046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36867d-a538-4752-b76e-8070a3e071df None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:37.805324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36867d-a538-4752-b76e-8070a3e071df None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:37.805597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36867d-a538-4752-b76e-8070a3e071df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:37.809418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:37.818927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36867d-a538-4752-b76e-8070a3e071df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:37.820554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36867d-a538-4752-b76e-8070a3e071df None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-3a36867d-a538-4752-b76e-8070a3e071df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:37.839973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:37.840968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:37.841932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:37.841932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:37.841932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:37.841932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:37.841932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 290/563] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:37.843151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36867d-a538-4752-b76e-8070a3e071df None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:37.856645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:37.876146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 274/564] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:37.877055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae8792-ee84-4033-9e89-69a32b264b9b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:37.982826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36867d-a538-4752-b76e-8070a3e071df None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:37.997257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae8792-ee84-4033-9e89-69a32b264b9b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:38.028627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064859c-a9ad-4083-a48a-eac83052fdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:38.029317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064859c-a9ad-4083-a48a-eac83052fdc1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:38.029597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064859c-a9ad-4083-a48a-eac83052fdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:38.029905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064859c-a9ad-4083-a48a-eac83052fdc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:38.030682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064859c-a9ad-4083-a48a-eac83052fdc1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:38.030999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064859c-a9ad-4083-a48a-eac83052fdc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:38.032431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064859c-a9ad-4083-a48a-eac83052fdc1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:38.039518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a9b61-6ee7-4111-ad51-26e1f1b74015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:38.040353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a9b61-6ee7-4111-ad51-26e1f1b74015 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:38.040637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a9b61-6ee7-4111-ad51-26e1f1b74015 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:38.040914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a9b61-6ee7-4111-ad51-26e1f1b74015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:38.049092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 291/565] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:38.049845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064859c-a9ad-4083-a48a-eac83052fdc1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:38.055181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a9b61-6ee7-4111-ad51-26e1f1b74015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:38.056685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a9b61-6ee7-4111-ad51-26e1f1b74015 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-af4a9b61-6ee7-4111-ad51-26e1f1b74015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:38.075777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:38.076778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:38.077852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:38.077852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:38.077852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:38.077852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:38.077852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 275/566] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:38.077852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a9b61-6ee7-4111-ad51-26e1f1b74015 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:38.089578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:38.148425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:38.194266 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064859c-a9ad-4083-a48a-eac83052fdc1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:38.207783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a9b61-6ee7-4111-ad51-26e1f1b74015 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:38.243376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16742ec-bb57-4549-b362-9e7502ca790c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:38.244147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16742ec-bb57-4549-b362-9e7502ca790c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:38.244336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16742ec-bb57-4549-b362-9e7502ca790c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:38.244613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16742ec-bb57-4549-b362-9e7502ca790c None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:38.245355 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16742ec-bb57-4549-b362-9e7502ca790c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:38.245689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16742ec-bb57-4549-b362-9e7502ca790c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:38.247118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16742ec-bb57-4549-b362-9e7502ca790c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:38.255964 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaf56a-4992-474a-9c01-751c5f39b095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:38.256782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaf56a-4992-474a-9c01-751c5f39b095 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:38.257226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaf56a-4992-474a-9c01-751c5f39b095 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:38.257759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaf56a-4992-474a-9c01-751c5f39b095 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:38.262745 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 292/567] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:38.264086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416df07-2624-4599-81ce-51cb64f7b7b9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:38.264374 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416df07-2624-4599-81ce-51cb64f7b7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:38.264651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416df07-2624-4599-81ce-51cb64f7b7b9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:38.266106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 293/568] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:38.266609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416df07-2624-4599-81ce-51cb64f7b7b9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:38.271625 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcaf56a-4992-474a-9c01-751c5f39b095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-3dcaf56a-4992-474a-9c01-751c5f39b095 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:38.279820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:38.280875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:38.281744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 276/569] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:38.282462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaf56a-4992-474a-9c01-751c5f39b095 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:38.360789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:38.401225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaf56a-4992-474a-9c01-751c5f39b095 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:38.409697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416df07-2624-4599-81ce-51cb64f7b7b9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:38.440420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044bcc6-ebd5-44c4-8415-c6ea48973e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:38.441091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044bcc6-ebd5-44c4-8415-c6ea48973e36 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:38.441369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044bcc6-ebd5-44c4-8415-c6ea48973e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:38.441669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044bcc6-ebd5-44c4-8415-c6ea48973e36 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:38.442325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:38.442433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044bcc6-ebd5-44c4-8415-c6ea48973e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:38.442702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044bcc6-ebd5-44c4-8415-c6ea48973e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:38.444272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044bcc6-ebd5-44c4-8415-c6ea48973e36 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:38.450137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c023ad-55cf-4778-a2ac-9fe82b5ca78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:38.450949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c023ad-55cf-4778-a2ac-9fe82b5ca78d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:38.451246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c023ad-55cf-4778-a2ac-9fe82b5ca78d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:38.451509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c023ad-55cf-4778-a2ac-9fe82b5ca78d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:38.461520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 277/570] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:38.462607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044bcc6-ebd5-44c4-8415-c6ea48973e36 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:38.468439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c023ad-55cf-4778-a2ac-9fe82b5ca78d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:38.469981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c023ad-55cf-4778-a2ac-9fe82b5ca78d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-03c023ad-55cf-4778-a2ac-9fe82b5ca78d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:38.490900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:38.492323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:38.493499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:38.493499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:38.493499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:38.493499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:38.493499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 294/571] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:38.494256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f807329-7d4e-4c62-a5b3-27dfd1626da2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:38.494539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f807329-7d4e-4c62-a5b3-27dfd1626da2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:38.494817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f807329-7d4e-4c62-a5b3-27dfd1626da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:38.514005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7f807329-7d4e-4c62-a5b3-27dfd1626da2 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:38.515214 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7f807329-7d4e-4c62-a5b3-27dfd1626da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:38.588006 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044bcc6-ebd5-44c4-8415-c6ea48973e36 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:38.596273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 295/572] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:38.597145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f807329-7d4e-4c62-a5b3-27dfd1626da2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:38.623918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:38.635356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb6f84-2830-4822-b2e6-618ec479e000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:38.636207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb6f84-2830-4822-b2e6-618ec479e000 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:38.636488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb6f84-2830-4822-b2e6-618ec479e000 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:38.636760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb6f84-2830-4822-b2e6-618ec479e000 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:38.644316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb6f84-2830-4822-b2e6-618ec479e000 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:38.645821 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb6f84-2830-4822-b2e6-618ec479e000 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-aafb6f84-2830-4822-b2e6-618ec479e000 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:38.662818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:38.664005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:38.665197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:38.665197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:38.665197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:38.665197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:38.665197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:38.665197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:38.665197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:38.665197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:38.665197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 278/573] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:38.665969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb6f84-2830-4822-b2e6-618ec479e000 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:38.726795 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f807329-7d4e-4c62-a5b3-27dfd1626da2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:38.741661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:38.766279 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc024e6-1f54-4674-919f-daa6a2f1c1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:38.767053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc024e6-1f54-4674-919f-daa6a2f1c1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:38.767237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc024e6-1f54-4674-919f-daa6a2f1c1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:38.767513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc024e6-1f54-4674-919f-daa6a2f1c1f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:38.768278 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc024e6-1f54-4674-919f-daa6a2f1c1f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:38.768585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc024e6-1f54-4674-919f-daa6a2f1c1f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:38.770009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc024e6-1f54-4674-919f-daa6a2f1c1f3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:38.779360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 296/574] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:38.780044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc024e6-1f54-4674-919f-daa6a2f1c1f3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:38.790865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb6f84-2830-4822-b2e6-618ec479e000 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:38.830171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0051789-4fd3-4701-8ded-559f64fc8c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:38.830985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0051789-4fd3-4701-8ded-559f64fc8c28 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:38.831270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0051789-4fd3-4701-8ded-559f64fc8c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:38.831542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0051789-4fd3-4701-8ded-559f64fc8c28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:38.839830 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0051789-4fd3-4701-8ded-559f64fc8c28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:38.841322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0051789-4fd3-4701-8ded-559f64fc8c28 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a0051789-4fd3-4701-8ded-559f64fc8c28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:38.857028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:38.858456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:38.860103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:38.860103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:38.860103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:38.860103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:38.860103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:38.860103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:38.860103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:38.860103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:38.860103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 279/575] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:38.860103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0051789-4fd3-4701-8ded-559f64fc8c28 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:38.899525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc024e6-1f54-4674-919f-daa6a2f1c1f3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:38.926941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:38.939714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958c7c-4610-4e82-b373-f8c643e9ba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:38.940389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958c7c-4610-4e82-b373-f8c643e9ba6d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:38.940674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958c7c-4610-4e82-b373-f8c643e9ba6d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:38.940953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958c7c-4610-4e82-b373-f8c643e9ba6d None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:38.941711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958c7c-4610-4e82-b373-f8c643e9ba6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:38.942026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958c7c-4610-4e82-b373-f8c643e9ba6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:38.943419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958c7c-4610-4e82-b373-f8c643e9ba6d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:38.957105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 297/576] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:38.957985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958c7c-4610-4e82-b373-f8c643e9ba6d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:38.988737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0051789-4fd3-4701-8ded-559f64fc8c28 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:39.002544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:39.030433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb0e16-8b84-47ee-902d-69d018ad75d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:39.031237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb0e16-8b84-47ee-902d-69d018ad75d0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:39.031520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb0e16-8b84-47ee-902d-69d018ad75d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:39.031795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb0e16-8b84-47ee-902d-69d018ad75d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:39.044187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb0e16-8b84-47ee-902d-69d018ad75d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9bfb0e16-8b84-47ee-902d-69d018ad75d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:39.052189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:39.053299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:39.054205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 280/577] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:39.055046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb0e16-8b84-47ee-902d-69d018ad75d0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:39.105825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958c7c-4610-4e82-b373-f8c643e9ba6d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:39.155973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6013bff-3549-41b4-9721-651219660c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:39.156643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6013bff-3549-41b4-9721-651219660c2a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:39.156905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6013bff-3549-41b4-9721-651219660c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:39.157176 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6013bff-3549-41b4-9721-651219660c2a None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:39.158024 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6013bff-3549-41b4-9721-651219660c2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:39.158334 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6013bff-3549-41b4-9721-651219660c2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:39.159784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6013bff-3549-41b4-9721-651219660c2a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:39.176472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 298/578] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:39.176472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6013bff-3549-41b4-9721-651219660c2a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:39.188008 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb0e16-8b84-47ee-902d-69d018ad75d0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:39.198632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:39.228534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56173cd6-07b9-42d1-b4df-689d31e7e55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:39.229342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56173cd6-07b9-42d1-b4df-689d31e7e55c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:39.229640 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56173cd6-07b9-42d1-b4df-689d31e7e55c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:39.229947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56173cd6-07b9-42d1-b4df-689d31e7e55c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:39.242491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56173cd6-07b9-42d1-b4df-689d31e7e55c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:39.244015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56173cd6-07b9-42d1-b4df-689d31e7e55c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-56173cd6-07b9-42d1-b4df-689d31e7e55c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:39.264677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:39.266565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:39.268450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:39.268450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:39.268450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:39.268450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:39.268450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 281/579] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:39.268450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56173cd6-07b9-42d1-b4df-689d31e7e55c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:39.297190 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:39.308695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6013bff-3549-41b4-9721-651219660c2a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:39.360806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c479ee-9d6a-4f5a-bb43-c108e067bbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:39.361478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c479ee-9d6a-4f5a-bb43-c108e067bbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:39.361797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c479ee-9d6a-4f5a-bb43-c108e067bbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:39.362073 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c479ee-9d6a-4f5a-bb43-c108e067bbb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:39.362866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c479ee-9d6a-4f5a-bb43-c108e067bbb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:39.363181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c479ee-9d6a-4f5a-bb43-c108e067bbb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:39.364657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c479ee-9d6a-4f5a-bb43-c108e067bbb6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:39.378054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 299/580] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:39.378906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c479ee-9d6a-4f5a-bb43-c108e067bbb6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:39.381511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56173cd6-07b9-42d1-b4df-689d31e7e55c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:39.418368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5094d75-78d0-4c93-b39c-37229f713c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:39.419371 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5094d75-78d0-4c93-b39c-37229f713c08 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:39.419742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5094d75-78d0-4c93-b39c-37229f713c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:39.420038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5094d75-78d0-4c93-b39c-37229f713c08 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:39.429457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5094d75-78d0-4c93-b39c-37229f713c08 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:39.430970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5094d75-78d0-4c93-b39c-37229f713c08 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-d5094d75-78d0-4c93-b39c-37229f713c08 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:39.446343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:39.447652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:39.450284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:39.450284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:39.450284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:39.450284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:39.450284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:39.450284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:39.450284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:39.450284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:39.450284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 07:50:39.450284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:39.450284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 282/581] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:39.450284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5094d75-78d0-4c93-b39c-37229f713c08 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:39.479337 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:39.503033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c479ee-9d6a-4f5a-bb43-c108e067bbb6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:39.542316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d420b-19b5-4173-adf1-75a1ec613619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:39.542997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d420b-19b5-4173-adf1-75a1ec613619 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:39.543280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d420b-19b5-4173-adf1-75a1ec613619 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:39.543556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d420b-19b5-4173-adf1-75a1ec613619 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:39.544317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d420b-19b5-4173-adf1-75a1ec613619 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:39.544628 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d420b-19b5-4173-adf1-75a1ec613619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:39.546228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d420b-19b5-4173-adf1-75a1ec613619 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:39.556080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 300/582] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:39.556937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d420b-19b5-4173-adf1-75a1ec613619 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:39.577080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:39.587865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5094d75-78d0-4c93-b39c-37229f713c08 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:39.627781 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bfd62-4e1a-46e3-8c96-7b03a6f37eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:39.628428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bfd62-4e1a-46e3-8c96-7b03a6f37eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:39.628752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bfd62-4e1a-46e3-8c96-7b03a6f37eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:39.629358 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bfd62-4e1a-46e3-8c96-7b03a6f37eb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:39.630276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bfd62-4e1a-46e3-8c96-7b03a6f37eb4 None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=6763884b51204c488ca8c7e452e3a29e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:39.630593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bfd62-4e1a-46e3-8c96-7b03a6f37eb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:39.632193 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bfd62-4e1a-46e3-8c96-7b03a6f37eb4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:39.638794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 283/583] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:39 2022] GET /identity/v3/role_assignments?user.id=6763884b51204c488ca8c7e452e3a29e&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:39.640281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bfd62-4e1a-46e3-8c96-7b03a6f37eb4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:39.682034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d420b-19b5-4173-adf1-75a1ec613619 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:39.728861 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4cdd2-c568-4514-950f-90390eed311a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:39.729562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4cdd2-c568-4514-950f-90390eed311a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:39.729886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4cdd2-c568-4514-950f-90390eed311a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:39.730190 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4cdd2-c568-4514-950f-90390eed311a None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:39.730946 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4cdd2-c568-4514-950f-90390eed311a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:39.731254 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4cdd2-c568-4514-950f-90390eed311a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:39.732713 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4cdd2-c568-4514-950f-90390eed311a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:39.749596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 301/584] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:39.750548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4cdd2-c568-4514-950f-90390eed311a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:39.781815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:39.805323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bfd62-4e1a-46e3-8c96-7b03a6f37eb4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:39.847151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613a9cb-14e0-40ac-855a-c31e395e6ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:39.848137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613a9cb-14e0-40ac-855a-c31e395e6ad2 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:39.848488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613a9cb-14e0-40ac-855a-c31e395e6ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:39.848773 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613a9cb-14e0-40ac-855a-c31e395e6ad2 None admin] PATH_INFO: `/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/15211e42ec9d4c1699d877f0b55ce918/roles/fd54d351b4ac49d7be6d0690309c00e7` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:39.866870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613a9cb-14e0-40ac-855a-c31e395e6ad2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f2d85c9e3d14bdabfb63fe505d83714, user_id=15211e42ec9d4c1699d877f0b55ce918, role_id=fd54d351b4ac49d7be6d0690309c00e7)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:39.868490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613a9cb-14e0-40ac-855a-c31e395e6ad2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:39.880261 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:39.893657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 284/585] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:50:39 2022] PUT /identity/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/15211e42ec9d4c1699d877f0b55ce918/roles/fd54d351b4ac49d7be6d0690309c00e7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:50:39.893657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613a9cb-14e0-40ac-855a-c31e395e6ad2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:39.935794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4cdd2-c568-4514-950f-90390eed311a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:39.993851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d5c3a-14e5-43b2-b303-d46489542260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:39.994908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d5c3a-14e5-43b2-b303-d46489542260 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:39.995260 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d5c3a-14e5-43b2-b303-d46489542260 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:39.995613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d5c3a-14e5-43b2-b303-d46489542260 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:39.996558 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d5c3a-14e5-43b2-b303-d46489542260 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:39.997341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d5c3a-14e5-43b2-b303-d46489542260 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:40.000274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d5c3a-14e5-43b2-b303-d46489542260 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:40.029832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 302/586] 10.222.0.43 () {56 vars in 1137 bytes} [Mon Jul 4 07:50:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:40.030998 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d5c3a-14e5-43b2-b303-d46489542260 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:40.046374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613a9cb-14e0-40ac-855a-c31e395e6ad2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:40.092716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb2b0b-3564-4759-a18d-bc6c845f9751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:40.093874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb2b0b-3564-4759-a18d-bc6c845f9751 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:40.094308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb2b0b-3564-4759-a18d-bc6c845f9751 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:40.094729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb2b0b-3564-4759-a18d-bc6c845f9751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:40.097424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:40.110905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb2b0b-3564-4759-a18d-bc6c845f9751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:40.113686 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb2b0b-3564-4759-a18d-bc6c845f9751 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-95fb2b0b-3564-4759-a18d-bc6c845f9751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:40.138662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:40.139656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:40.140568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:40.140568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:40.140568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:40.140568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:40.141221 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 285/587] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:40.142089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb2b0b-3564-4759-a18d-bc6c845f9751 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:40.170153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d5c3a-14e5-43b2-b303-d46489542260 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:40.224053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192d9c2-a1a7-497d-b28a-dcda8930afb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:40.225129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192d9c2-a1a7-497d-b28a-dcda8930afb6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:40.225511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192d9c2-a1a7-497d-b28a-dcda8930afb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:40.226036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192d9c2-a1a7-497d-b28a-dcda8930afb6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:40.228326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:40.236461 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192d9c2-a1a7-497d-b28a-dcda8930afb6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:40.238078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192d9c2-a1a7-497d-b28a-dcda8930afb6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e192d9c2-a1a7-497d-b28a-dcda8930afb6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:40.255954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:40.259042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:40.261324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:40.261324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:40.261324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:40.261324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:40.261324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:40.261324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:40.261324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:40.261324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:40.261324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 07:50:40.261324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:40.261324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 303/588] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:40.261324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192d9c2-a1a7-497d-b28a-dcda8930afb6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:40.285089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb2b0b-3564-4759-a18d-bc6c845f9751 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:40.321243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b41a6-0225-494c-8f17-bb4346f7d42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:40.322142 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b41a6-0225-494c-8f17-bb4346f7d42b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:40.322422 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b41a6-0225-494c-8f17-bb4346f7d42b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:40.322739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b41a6-0225-494c-8f17-bb4346f7d42b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:40.333761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b41a6-0225-494c-8f17-bb4346f7d42b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:40.335305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b41a6-0225-494c-8f17-bb4346f7d42b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9f7b41a6-0225-494c-8f17-bb4346f7d42b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:40.355636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:40.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:40.359638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:40.359638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:40.359638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:40.359638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:40.359638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 286/589] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:40.359638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b41a6-0225-494c-8f17-bb4346f7d42b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:40.393031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192d9c2-a1a7-497d-b28a-dcda8930afb6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:40.420632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:40.441551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d74a3-35d1-42d7-beba-fd384c129d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:40.442862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d74a3-35d1-42d7-beba-fd384c129d50 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:40.443452 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d74a3-35d1-42d7-beba-fd384c129d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:40.444108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d74a3-35d1-42d7-beba-fd384c129d50 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:40.445026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d74a3-35d1-42d7-beba-fd384c129d50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:40.445486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d74a3-35d1-42d7-beba-fd384c129d50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:40.447490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d74a3-35d1-42d7-beba-fd384c129d50 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:40.465022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 304/590] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:40.466093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d74a3-35d1-42d7-beba-fd384c129d50 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:40.510732 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b41a6-0225-494c-8f17-bb4346f7d42b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:40.544185 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:40.557031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36005ed1-efbe-4d28-9bb9-4fe6c4658fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:40.557884 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36005ed1-efbe-4d28-9bb9-4fe6c4658fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:40.558226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36005ed1-efbe-4d28-9bb9-4fe6c4658fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:40.558565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36005ed1-efbe-4d28-9bb9-4fe6c4658fe7 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:40.559490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36005ed1-efbe-4d28-9bb9-4fe6c4658fe7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:40.559867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36005ed1-efbe-4d28-9bb9-4fe6c4658fe7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:40.561636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36005ed1-efbe-4d28-9bb9-4fe6c4658fe7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:40.588013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 287/591] 10.222.0.43 () {56 vars in 1137 bytes} [Mon Jul 4 07:50:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:40.588013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36005ed1-efbe-4d28-9bb9-4fe6c4658fe7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:40.616577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d74a3-35d1-42d7-beba-fd384c129d50 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:40.670587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b3afb-5f2d-43fa-ae45-691a97d0952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:40.671915 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b3afb-5f2d-43fa-ae45-691a97d0952f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:40.672402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b3afb-5f2d-43fa-ae45-691a97d0952f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:40.673859 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b3afb-5f2d-43fa-ae45-691a97d0952f None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:40.674800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b3afb-5f2d-43fa-ae45-691a97d0952f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:40.675285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b3afb-5f2d-43fa-ae45-691a97d0952f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:40.677255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b3afb-5f2d-43fa-ae45-691a97d0952f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:40.700095 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 305/592] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:40.701568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b3afb-5f2d-43fa-ae45-691a97d0952f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:40.708775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:40.716965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36005ed1-efbe-4d28-9bb9-4fe6c4658fe7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:40.768173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b17caa-6218-483a-8894-9b491fdad5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:40.769450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b17caa-6218-483a-8894-9b491fdad5af None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:40.769822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b17caa-6218-483a-8894-9b491fdad5af None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:40.770167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b17caa-6218-483a-8894-9b491fdad5af None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:40.781332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b17caa-6218-483a-8894-9b491fdad5af None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:40.783843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b17caa-6218-483a-8894-9b491fdad5af None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-53b17caa-6218-483a-8894-9b491fdad5af None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:40.807214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:40.808757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:40.810002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:40.810002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:40.810002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:40.810002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:40.810002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:40.810002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:40.810002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:40.810002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:40.810002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 07:50:40.810002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:40.810002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 288/593] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:40.810639 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b17caa-6218-483a-8894-9b491fdad5af None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:40.843731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b3afb-5f2d-43fa-ae45-691a97d0952f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:40.862238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:40.889970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb0863-6fa2-473d-86af-3d1dddad82bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:40.891046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb0863-6fa2-473d-86af-3d1dddad82bd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:40.891507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb0863-6fa2-473d-86af-3d1dddad82bd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:40.892022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb0863-6fa2-473d-86af-3d1dddad82bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:40.906851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb0863-6fa2-473d-86af-3d1dddad82bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:40.909206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb0863-6fa2-473d-86af-3d1dddad82bd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8feb0863-6fa2-473d-86af-3d1dddad82bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:40.934930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:40.936101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:40.937119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:40.937119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:40.937119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:40.937119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:40.941963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 306/594] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:40.943216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb0863-6fa2-473d-86af-3d1dddad82bd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:40.961596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b17caa-6218-483a-8894-9b491fdad5af None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:41.008224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96a7a0-251c-4ba0-968f-c5dab6080215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:41.009064 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96a7a0-251c-4ba0-968f-c5dab6080215 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:41.009343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96a7a0-251c-4ba0-968f-c5dab6080215 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:41.009614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96a7a0-251c-4ba0-968f-c5dab6080215 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:41.019314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96a7a0-251c-4ba0-968f-c5dab6080215 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:41.021348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96a7a0-251c-4ba0-968f-c5dab6080215 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9e96a7a0-251c-4ba0-968f-c5dab6080215 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:41.041956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:41.043127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:41.044131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:41.044131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:41.044131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:41.044131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:41.044131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:41.044131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:41.044131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:41.044131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:41.050001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 289/595] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:41.051014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96a7a0-251c-4ba0-968f-c5dab6080215 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:41.079767 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:41.082987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb0863-6fa2-473d-86af-3d1dddad82bd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:41.135411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4af608-c982-47e5-9fae-73ca3840d03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:41.137560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4af608-c982-47e5-9fae-73ca3840d03d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:41.138198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4af608-c982-47e5-9fae-73ca3840d03d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:41.138786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4af608-c982-47e5-9fae-73ca3840d03d None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:41.141216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4af608-c982-47e5-9fae-73ca3840d03d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:41.142293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4af608-c982-47e5-9fae-73ca3840d03d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:41.144144 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4af608-c982-47e5-9fae-73ca3840d03d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:41.168175 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 307/596] 10.222.0.43 () {56 vars in 1139 bytes} [Mon Jul 4 07:50:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:41.169182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4af608-c982-47e5-9fae-73ca3840d03d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:41.208427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:41.215534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96a7a0-251c-4ba0-968f-c5dab6080215 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:41.261200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e30f9-9a7a-4d85-befb-4c344f19af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:41.261992 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e30f9-9a7a-4d85-befb-4c344f19af56 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:41.262373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e30f9-9a7a-4d85-befb-4c344f19af56 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:41.262752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e30f9-9a7a-4d85-befb-4c344f19af56 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:41.263582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e30f9-9a7a-4d85-befb-4c344f19af56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:41.264013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e30f9-9a7a-4d85-befb-4c344f19af56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:41.265484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e30f9-9a7a-4d85-befb-4c344f19af56 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:41.283423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 290/597] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:41.284454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e30f9-9a7a-4d85-befb-4c344f19af56 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:41.327636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4af608-c982-47e5-9fae-73ca3840d03d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:41.365077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:41.387778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb71aba-478c-45f9-9915-107d7abae037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:41.388806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb71aba-478c-45f9-9915-107d7abae037 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:41.389225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb71aba-478c-45f9-9915-107d7abae037 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:41.389683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb71aba-478c-45f9-9915-107d7abae037 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:41.390651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb71aba-478c-45f9-9915-107d7abae037 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:41.391100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb71aba-478c-45f9-9915-107d7abae037 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:41.391228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e30f9-9a7a-4d85-befb-4c344f19af56 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:41.393502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb71aba-478c-45f9-9915-107d7abae037 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:41.409910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 308/598] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:41.411026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb71aba-478c-45f9-9915-107d7abae037 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:41.445534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb3f66-a968-45c0-abd5-e004962ff46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:41.448407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb3f66-a968-45c0-abd5-e004962ff46e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:41.449417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb3f66-a968-45c0-abd5-e004962ff46e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:41.450412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb3f66-a968-45c0-abd5-e004962ff46e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:41.470555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb3f66-a968-45c0-abd5-e004962ff46e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:41.473187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb3f66-a968-45c0-abd5-e004962ff46e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-f3fb3f66-a968-45c0-abd5-e004962ff46e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:41.499825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:41.500815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:41.501760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:41.501760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:41.501760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:41.501760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:41.504631 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 291/599] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:41.505904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb3f66-a968-45c0-abd5-e004962ff46e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:41.523825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:41.540653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb71aba-478c-45f9-9915-107d7abae037 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:41.576568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cf921-738c-4648-b58e-1cf839a82706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:41.577480 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cf921-738c-4648-b58e-1cf839a82706 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:41.577867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cf921-738c-4648-b58e-1cf839a82706 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:41.578205 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cf921-738c-4648-b58e-1cf839a82706 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:41.586138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19cf921-738c-4648-b58e-1cf839a82706 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:41.587503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19cf921-738c-4648-b58e-1cf839a82706 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d19cf921-738c-4648-b58e-1cf839a82706 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:41.602937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:41.603936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:41.604854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:41.604854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:41.604854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:41.604854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:41.604854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:41.604854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:41.604854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:41.604854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:41.608867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 309/600] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:41.609927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cf921-738c-4648-b58e-1cf839a82706 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:41.626556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb3f66-a968-45c0-abd5-e004962ff46e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:41.657384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:41.668291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75710c-1404-45de-9b33-27c33bea83e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:41.669725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75710c-1404-45de-9b33-27c33bea83e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:41.670290 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75710c-1404-45de-9b33-27c33bea83e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:41.670820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75710c-1404-45de-9b33-27c33bea83e7 None admin] PATH_INFO: `/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/061ffcf242ff4155b92daa8c3888fd10/roles/fd54d351b4ac49d7be6d0690309c00e7` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:41.688066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75710c-1404-45de-9b33-27c33bea83e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f2d85c9e3d14bdabfb63fe505d83714, user_id=061ffcf242ff4155b92daa8c3888fd10, role_id=fd54d351b4ac49d7be6d0690309c00e7)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:41.689995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75710c-1404-45de-9b33-27c33bea83e7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:41.707104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 292/601] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:50:41 2022] PUT /identity/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/061ffcf242ff4155b92daa8c3888fd10/roles/fd54d351b4ac49d7be6d0690309c00e7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:50:41.708326 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75710c-1404-45de-9b33-27c33bea83e7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:41.730254 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cf921-738c-4648-b58e-1cf839a82706 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:41.812192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee791304-3d5e-4b57-af8a-953d3006fd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:41.812804 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee791304-3d5e-4b57-af8a-953d3006fd68 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:41.813096 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee791304-3d5e-4b57-af8a-953d3006fd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:41.813470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee791304-3d5e-4b57-af8a-953d3006fd68 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:41.814327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee791304-3d5e-4b57-af8a-953d3006fd68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:41.814665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee791304-3d5e-4b57-af8a-953d3006fd68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:41.816156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee791304-3d5e-4b57-af8a-953d3006fd68 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:41.830424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 310/602] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:41.831882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90eb47-ecc9-432c-8b66-eff5781c70b5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:41.832174 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90eb47-ecc9-432c-8b66-eff5781c70b5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:41.832460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90eb47-ecc9-432c-8b66-eff5781c70b5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:41.834128 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 311/603] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:41.840088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90eb47-ecc9-432c-8b66-eff5781c70b5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:41.844300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:41.854391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75710c-1404-45de-9b33-27c33bea83e7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:41.899822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fad0a-49ac-4d90-846a-8d466be444be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:41.900557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fad0a-49ac-4d90-846a-8d466be444be None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:41.900923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fad0a-49ac-4d90-846a-8d466be444be None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:41.901878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fad0a-49ac-4d90-846a-8d466be444be None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:41.902784 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fad0a-49ac-4d90-846a-8d466be444be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:41.903108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fad0a-49ac-4d90-846a-8d466be444be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:41.905539 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fad0a-49ac-4d90-846a-8d466be444be None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:41.925560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 293/604] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:41.927721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbeb43-c94f-4bc7-90ee-adc30021ee7f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:41.927721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbeb43-c94f-4bc7-90ee-adc30021ee7f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:41.927721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbeb43-c94f-4bc7-90ee-adc30021ee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:41.949924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a0fbeb43-c94f-4bc7-90ee-adc30021ee7f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:41.951240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a0fbeb43-c94f-4bc7-90ee-adc30021ee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:41.982768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90eb47-ecc9-432c-8b66-eff5781c70b5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:42.002445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:42.032655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de911a0-5690-4fb7-bafb-927d556de261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:42.034568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de911a0-5690-4fb7-bafb-927d556de261 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:42.035382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de911a0-5690-4fb7-bafb-927d556de261 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:42.035866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de911a0-5690-4fb7-bafb-927d556de261 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:42.045653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 294/605] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:42.046516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbeb43-c94f-4bc7-90ee-adc30021ee7f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:42.046884 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de911a0-5690-4fb7-bafb-927d556de261 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:42.048651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de911a0-5690-4fb7-bafb-927d556de261 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8de911a0-5690-4fb7-bafb-927d556de261 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:42.066784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:42.067758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:42.068670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:42.068670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:42.068670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:42.068670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:42.068670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:42.068670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:42.068670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:42.068670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:42.071860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 312/606] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:42.077679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de911a0-5690-4fb7-bafb-927d556de261 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:42.164189 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:42.195506 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbeb43-c94f-4bc7-90ee-adc30021ee7f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:42.206624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de911a0-5690-4fb7-bafb-927d556de261 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:42.234221 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf1ec2-8e76-49b2-a144-776ff617080e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:42.235129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf1ec2-8e76-49b2-a144-776ff617080e None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:42.235406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf1ec2-8e76-49b2-a144-776ff617080e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:42.235679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf1ec2-8e76-49b2-a144-776ff617080e None admin] PATH_INFO: `/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/4a122422a45446c9a63d67b4f931fa47/roles/fd54d351b4ac49d7be6d0690309c00e7` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:42.247328 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98aa090-8e2e-4299-ba47-6cdf712a3d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:42.248296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98aa090-8e2e-4299-ba47-6cdf712a3d82 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:42.248758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98aa090-8e2e-4299-ba47-6cdf712a3d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:42.249029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98aa090-8e2e-4299-ba47-6cdf712a3d82 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:42.249869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98aa090-8e2e-4299-ba47-6cdf712a3d82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:42.250182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98aa090-8e2e-4299-ba47-6cdf712a3d82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:42.251870 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98aa090-8e2e-4299-ba47-6cdf712a3d82 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:42.252129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf1ec2-8e76-49b2-a144-776ff617080e None admin] RBAC: Authorizing `identity:create_grant(project_id=1f2d85c9e3d14bdabfb63fe505d83714, user_id=4a122422a45446c9a63d67b4f931fa47, role_id=fd54d351b4ac49d7be6d0690309c00e7)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:42.253567 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf1ec2-8e76-49b2-a144-776ff617080e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:42.266729 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 313/607] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:42.268194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98aa090-8e2e-4299-ba47-6cdf712a3d82 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:42.269505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 295/608] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:50:42 2022] PUT /identity/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/4a122422a45446c9a63d67b4f931fa47/roles/fd54d351b4ac49d7be6d0690309c00e7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:50:42.270881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a576965-cc0f-4104-9287-8fabea3e55f8 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:42.271162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a576965-cc0f-4104-9287-8fabea3e55f8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:42.271433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a576965-cc0f-4104-9287-8fabea3e55f8 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:42.273033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 296/609] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:42.273663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a576965-cc0f-4104-9287-8fabea3e55f8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:42.310098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:42.398003 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98aa090-8e2e-4299-ba47-6cdf712a3d82 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:42.428210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:42.431054 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a576965-cc0f-4104-9287-8fabea3e55f8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:42.443256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee098a-480f-47bd-aad9-f2b5cab56d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:42.444208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee098a-480f-47bd-aad9-f2b5cab56d0d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:42.444634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee098a-480f-47bd-aad9-f2b5cab56d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:42.445014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee098a-480f-47bd-aad9-f2b5cab56d0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:42.459237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee098a-480f-47bd-aad9-f2b5cab56d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-81ee098a-480f-47bd-aad9-f2b5cab56d0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:42.467705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:42.469102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:42.470082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 314/610] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:42.470896 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84499caa-450d-49db-a518-18e9c29eccc7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:42.470982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84499caa-450d-49db-a518-18e9c29eccc7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:42.471250 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84499caa-450d-49db-a518-18e9c29eccc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:42.481742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5600b75-98f2-4151-b8fb-f6c979a5a64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:42.483464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5600b75-98f2-4151-b8fb-f6c979a5a64a None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:42.483907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5600b75-98f2-4151-b8fb-f6c979a5a64a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:42.484412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5600b75-98f2-4151-b8fb-f6c979a5a64a None admin] PATH_INFO: `/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/429e173890b84ee289abc9793794ddb9/roles/fd54d351b4ac49d7be6d0690309c00e7` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:42.490188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-84499caa-450d-49db-a518-18e9c29eccc7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:42.492574 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-84499caa-450d-49db-a518-18e9c29eccc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:42.516578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5600b75-98f2-4151-b8fb-f6c979a5a64a None admin] RBAC: Authorizing `identity:create_grant(project_id=1f2d85c9e3d14bdabfb63fe505d83714, user_id=429e173890b84ee289abc9793794ddb9, role_id=fd54d351b4ac49d7be6d0690309c00e7)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:42.520860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5600b75-98f2-4151-b8fb-f6c979a5a64a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:42.550027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 297/611] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:50:42 2022] PUT /identity/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/429e173890b84ee289abc9793794ddb9/roles/fd54d351b4ac49d7be6d0690309c00e7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:50:42.551959 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5600b75-98f2-4151-b8fb-f6c979a5a64a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:42.592999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 315/612] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:42.638453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84499caa-450d-49db-a518-18e9c29eccc7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:42.678193 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:42.715552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5600b75-98f2-4151-b8fb-f6c979a5a64a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:42.768920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb0107-7878-4d3f-9fa5-1e531f731bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:42.770063 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb0107-7878-4d3f-9fa5-1e531f731bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:42.770545 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb0107-7878-4d3f-9fa5-1e531f731bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:42.771067 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb0107-7878-4d3f-9fa5-1e531f731bd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:42.772044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb0107-7878-4d3f-9fa5-1e531f731bd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:42.772602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb0107-7878-4d3f-9fa5-1e531f731bd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:42.777496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb0107-7878-4d3f-9fa5-1e531f731bd9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:42.793468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 298/613] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:42.799638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb0107-7878-4d3f-9fa5-1e531f731bd9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:42.860930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84499caa-450d-49db-a518-18e9c29eccc7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:42.880464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:42.917649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b364cb1-18e1-4ac1-9598-d9cf32646c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:42.917649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b364cb1-18e1-4ac1-9598-d9cf32646c82 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:42.917649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b364cb1-18e1-4ac1-9598-d9cf32646c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:42.917649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b364cb1-18e1-4ac1-9598-d9cf32646c82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:42.942435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b364cb1-18e1-4ac1-9598-d9cf32646c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:42.958980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb0107-7878-4d3f-9fa5-1e531f731bd9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8b364cb1-18e1-4ac1-9598-d9cf32646c82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:42.966159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:42.967178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:42.968039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 316/614] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:42.973408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b364cb1-18e1-4ac1-9598-d9cf32646c82 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:42.992331 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:43.007362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42269cb-9d5c-40da-985a-eb6c707cd654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:43.008216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42269cb-9d5c-40da-985a-eb6c707cd654 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:43.008505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42269cb-9d5c-40da-985a-eb6c707cd654 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:43.008775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42269cb-9d5c-40da-985a-eb6c707cd654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:43.033728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42269cb-9d5c-40da-985a-eb6c707cd654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:43.035256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42269cb-9d5c-40da-985a-eb6c707cd654 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c42269cb-9d5c-40da-985a-eb6c707cd654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:43.075316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:43.076325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:43.077230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:43.077230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:43.077230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:43.077230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:43.077230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 299/615] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:43.082839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42269cb-9d5c-40da-985a-eb6c707cd654 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:43.217742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b364cb1-18e1-4ac1-9598-d9cf32646c82 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:43.218275 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42269cb-9d5c-40da-985a-eb6c707cd654 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:43.252324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38982ff-7ddd-416d-aa57-8bf46d463618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:43.253031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38982ff-7ddd-416d-aa57-8bf46d463618 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:43.253310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38982ff-7ddd-416d-aa57-8bf46d463618 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:43.253581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38982ff-7ddd-416d-aa57-8bf46d463618 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:43.254667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38982ff-7ddd-416d-aa57-8bf46d463618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:43.254977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38982ff-7ddd-416d-aa57-8bf46d463618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:43.256349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38982ff-7ddd-416d-aa57-8bf46d463618 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:43.270269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 300/616] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:43.277974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38982ff-7ddd-416d-aa57-8bf46d463618 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:43.311340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c532c-606f-4e1a-a80e-eb76bed5cbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:43.312049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c532c-606f-4e1a-a80e-eb76bed5cbcd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:43.312332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c532c-606f-4e1a-a80e-eb76bed5cbcd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:43.312609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c532c-606f-4e1a-a80e-eb76bed5cbcd None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:43.313394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c532c-606f-4e1a-a80e-eb76bed5cbcd None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:43.313756 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c532c-606f-4e1a-a80e-eb76bed5cbcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:43.315161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c532c-606f-4e1a-a80e-eb76bed5cbcd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:43.327548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:43.336712 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 317/617] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:43.342177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c532c-606f-4e1a-a80e-eb76bed5cbcd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:43.388654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38982ff-7ddd-416d-aa57-8bf46d463618 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:43.424168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48390546-8897-4519-bdfc-fb9659a93801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:43.424978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48390546-8897-4519-bdfc-fb9659a93801 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:43.425312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48390546-8897-4519-bdfc-fb9659a93801 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:43.425794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48390546-8897-4519-bdfc-fb9659a93801 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:43.433563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48390546-8897-4519-bdfc-fb9659a93801 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:43.435051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48390546-8897-4519-bdfc-fb9659a93801 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:43.442721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-48390546-8897-4519-bdfc-fb9659a93801 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:43.452855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:43.454051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:43.455238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:43.455238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:43.455238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:43.455238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:43.455238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:43.455238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:43.455238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:43.455238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:43.455238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 07:50:43.455238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:43.457449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 301/618] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:50:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:43.463459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48390546-8897-4519-bdfc-fb9659a93801 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:43.596146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c532c-606f-4e1a-a80e-eb76bed5cbcd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:43.623638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48390546-8897-4519-bdfc-fb9659a93801 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:43.638777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:43.664164 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481061a-971e-41e7-9a49-06f770b082c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:43.664996 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481061a-971e-41e7-9a49-06f770b082c3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:43.665307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481061a-971e-41e7-9a49-06f770b082c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:43.665580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481061a-971e-41e7-9a49-06f770b082c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:43.688810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481061a-971e-41e7-9a49-06f770b082c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:43.690775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481061a-971e-41e7-9a49-06f770b082c3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:43.699938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825631cd-a89b-4d5f-b795-ce98ca5441dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:43.700637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825631cd-a89b-4d5f-b795-ce98ca5441dd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:43.700913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825631cd-a89b-4d5f-b795-ce98ca5441dd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:43.701183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825631cd-a89b-4d5f-b795-ce98ca5441dd None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:43.702112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825631cd-a89b-4d5f-b795-ce98ca5441dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:43.702420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825631cd-a89b-4d5f-b795-ce98ca5441dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:43.703949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825631cd-a89b-4d5f-b795-ce98ca5441dd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0481061a-971e-41e7-9a49-06f770b082c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:43.722177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:43.723177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:43.724085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:43.724085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:43.724085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:43.724085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:43.733819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 318/619] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:43.734857 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481061a-971e-41e7-9a49-06f770b082c3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:43.737296 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 302/620] 10.222.0.43 () {56 vars in 1135 bytes} [Mon Jul 4 07:50:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:43.742820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825631cd-a89b-4d5f-b795-ce98ca5441dd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:43.847178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825631cd-a89b-4d5f-b795-ce98ca5441dd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:43.884685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f3805-24a3-41c7-98a6-1b9d4fc30cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:43.885485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f3805-24a3-41c7-98a6-1b9d4fc30cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:43.885812 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f3805-24a3-41c7-98a6-1b9d4fc30cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:43.886196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f3805-24a3-41c7-98a6-1b9d4fc30cd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:43.897243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f3805-24a3-41c7-98a6-1b9d4fc30cd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:43.899262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f3805-24a3-41c7-98a6-1b9d4fc30cd1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-d60f3805-24a3-41c7-98a6-1b9d4fc30cd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:43.920080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:43.921033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:43.921981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:43.921981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:43.921981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:43.921981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:43.921981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 303/621] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:43.939185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f3805-24a3-41c7-98a6-1b9d4fc30cd1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:43.955426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:43.961419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:43.976534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481061a-971e-41e7-9a49-06f770b082c3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:44.035446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe2514-a01f-4f4a-a250-1d483d1ca24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:44.036683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe2514-a01f-4f4a-a250-1d483d1ca24b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:44.037203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe2514-a01f-4f4a-a250-1d483d1ca24b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:44.037723 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe2514-a01f-4f4a-a250-1d483d1ca24b None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:44.039334 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe2514-a01f-4f4a-a250-1d483d1ca24b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:44.040139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe2514-a01f-4f4a-a250-1d483d1ca24b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:44.042026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe2514-a01f-4f4a-a250-1d483d1ca24b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:44.063253 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f3805-24a3-41c7-98a6-1b9d4fc30cd1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:44.063580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 319/622] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:44.064947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32da392-f810-4c27-a2f8-2b6a9b32d506 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:44.065237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32da392-f810-4c27-a2f8-2b6a9b32d506 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:44.065513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32da392-f810-4c27-a2f8-2b6a9b32d506 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:44.072033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 320/623] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:44.072033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32da392-f810-4c27-a2f8-2b6a9b32d506 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:44.117999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef89577-c00d-481f-91bb-2a72187756ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:44.117999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef89577-c00d-481f-91bb-2a72187756ae None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:44.117999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef89577-c00d-481f-91bb-2a72187756ae None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:44.117999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef89577-c00d-481f-91bb-2a72187756ae None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:44.118407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef89577-c00d-481f-91bb-2a72187756ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:44.118407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef89577-c00d-481f-91bb-2a72187756ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:44.119257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef89577-c00d-481f-91bb-2a72187756ae None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:44.136576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 304/624] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:44.138359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58fa3c-1e9b-4a3e-8ea8-c0a725643858 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:44.138702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58fa3c-1e9b-4a3e-8ea8-c0a725643858 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:44.138987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58fa3c-1e9b-4a3e-8ea8-c0a725643858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:44.157117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-dc58fa3c-1e9b-4a3e-8ea8-c0a725643858 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:44.158417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-dc58fa3c-1e9b-4a3e-8ea8-c0a725643858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:44.220980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32da392-f810-4c27-a2f8-2b6a9b32d506 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:44.253893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 305/625] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:44.255166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58fa3c-1e9b-4a3e-8ea8-c0a725643858 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:44.258362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:44.264488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f143d-4fcc-4623-bd32-2ae4bd144abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:44.266262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f143d-4fcc-4623-bd32-2ae4bd144abe None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:44.266753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f143d-4fcc-4623-bd32-2ae4bd144abe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:44.267244 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f143d-4fcc-4623-bd32-2ae4bd144abe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:44.276945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f143d-4fcc-4623-bd32-2ae4bd144abe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:44.278768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f143d-4fcc-4623-bd32-2ae4bd144abe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-164f143d-4fcc-4623-bd32-2ae4bd144abe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:44.298470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:44.299560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:44.300556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:44.300556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:44.300556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:44.300556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:44.300556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:44.300556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:44.300556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:44.300556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:44.300556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:50:44.300556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:44.303643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 321/626] 10.222.0.43 () {56 vars in 1093 bytes} [Mon Jul 4 07:50:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:44.305293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f143d-4fcc-4623-bd32-2ae4bd144abe None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:44.319890 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:44.404371 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58fa3c-1e9b-4a3e-8ea8-c0a725643858 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:44.419414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f143d-4fcc-4623-bd32-2ae4bd144abe None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:44.448380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae8bc8-c696-47c3-b57c-d10b768f72cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:44.450110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae8bc8-c696-47c3-b57c-d10b768f72cf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:44.450562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae8bc8-c696-47c3-b57c-d10b768f72cf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:44.451132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae8bc8-c696-47c3-b57c-d10b768f72cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:44.461867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae8bc8-c696-47c3-b57c-d10b768f72cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:44.463566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae8bc8-c696-47c3-b57c-d10b768f72cf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:44.468089 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac5bdb-ff98-4682-a1bc-aca6ac50bde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:44.469120 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac5bdb-ff98-4682-a1bc-aca6ac50bde3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:44.469546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac5bdb-ff98-4682-a1bc-aca6ac50bde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:44.469976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac5bdb-ff98-4682-a1bc-aca6ac50bde3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0eae8bc8-c696-47c3-b57c-d10b768f72cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:44.479613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:44.480779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:44.481913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:44.481913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:44.481913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:44.481913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:44.481913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:44.481913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:44.481913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:44.481913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:44.483907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 306/627] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:44.484860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae8bc8-c696-47c3-b57c-d10b768f72cf None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:44.485543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac5bdb-ff98-4682-a1bc-aca6ac50bde3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5dac5bdb-ff98-4682-a1bc-aca6ac50bde3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:44.495858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:44.497020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:44.498158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:44.498158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:44.498158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:44.498158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:44.498158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:44.498158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:44.498158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:44.498158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:44.498158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:44.498158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:44.498158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:44.498158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:44.501090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 322/628] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:44.502167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac5bdb-ff98-4682-a1bc-aca6ac50bde3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:44.560741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:44.595618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:44.606014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae8bc8-c696-47c3-b57c-d10b768f72cf None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:44.637275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac5bdb-ff98-4682-a1bc-aca6ac50bde3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:44.643697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663da215-b9f4-426e-acd0-e35b1cfb775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:44.644528 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663da215-b9f4-426e-acd0-e35b1cfb775e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:44.645259 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663da215-b9f4-426e-acd0-e35b1cfb775e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:44.645555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663da215-b9f4-426e-acd0-e35b1cfb775e None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:44.646452 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663da215-b9f4-426e-acd0-e35b1cfb775e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:44.646762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663da215-b9f4-426e-acd0-e35b1cfb775e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:44.649069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663da215-b9f4-426e-acd0-e35b1cfb775e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:44.678126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e329fd8-a2b8-428f-a47c-a0890ecb0473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:44.679742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e329fd8-a2b8-428f-a47c-a0890ecb0473 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:44.680122 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e329fd8-a2b8-428f-a47c-a0890ecb0473 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:44.680401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e329fd8-a2b8-428f-a47c-a0890ecb0473 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:44.681226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e329fd8-a2b8-428f-a47c-a0890ecb0473 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:44.681539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e329fd8-a2b8-428f-a47c-a0890ecb0473 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:44.687046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 307/629] 10.222.0.43 () {56 vars in 1133 bytes} [Mon Jul 4 07:50:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:44.687046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663da215-b9f4-426e-acd0-e35b1cfb775e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:44.687288 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e329fd8-a2b8-428f-a47c-a0890ecb0473 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:44.705193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 323/630] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:44.705193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a4a87-ef85-4d53-85b1-bf9e4aad2efe None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:44.705193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a4a87-ef85-4d53-85b1-bf9e4aad2efe None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:44.705193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a4a87-ef85-4d53-85b1-bf9e4aad2efe None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:44.705193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 324/631] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:44.705193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a4a87-ef85-4d53-85b1-bf9e4aad2efe None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:44.801686 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663da215-b9f4-426e-acd0-e35b1cfb775e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:44.824760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a4a87-ef85-4d53-85b1-bf9e4aad2efe None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:44.825157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:44.841360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7310c-441c-45c2-b7a4-5732e2e50180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:44.842085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7310c-441c-45c2-b7a4-5732e2e50180 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:44.842353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7310c-441c-45c2-b7a4-5732e2e50180 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:44.842643 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7310c-441c-45c2-b7a4-5732e2e50180 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:44.843381 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7310c-441c-45c2-b7a4-5732e2e50180 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:44.843686 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7310c-441c-45c2-b7a4-5732e2e50180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:44.845209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7310c-441c-45c2-b7a4-5732e2e50180 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:44.859495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 308/632] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:44.859495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7310c-441c-45c2-b7a4-5732e2e50180 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:44.861464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:44.871928 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e0f0a-aaf8-40fd-93dc-bb4c883f87c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:44.872652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e0f0a-aaf8-40fd-93dc-bb4c883f87c0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:44.872914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e0f0a-aaf8-40fd-93dc-bb4c883f87c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:44.873187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e0f0a-aaf8-40fd-93dc-bb4c883f87c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:44.950777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e0f0a-aaf8-40fd-93dc-bb4c883f87c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:44.952549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e0f0a-aaf8-40fd-93dc-bb4c883f87c0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:44.971774 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7310c-441c-45c2-b7a4-5732e2e50180 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:45.019161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b347186-58c8-4cc5-816e-019f3f0fe8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:45.020559 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b347186-58c8-4cc5-816e-019f3f0fe8d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:45.020987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b347186-58c8-4cc5-816e-019f3f0fe8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:45.021264 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b347186-58c8-4cc5-816e-019f3f0fe8d3 None admin] PATH_INFO: `/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/6763884b51204c488ca8c7e452e3a29e/roles/fd54d351b4ac49d7be6d0690309c00e7` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:45.037496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b347186-58c8-4cc5-816e-019f3f0fe8d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f2d85c9e3d14bdabfb63fe505d83714, user_id=6763884b51204c488ca8c7e452e3a29e, role_id=fd54d351b4ac49d7be6d0690309c00e7)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:45.039054 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b347186-58c8-4cc5-816e-019f3f0fe8d3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:45.052998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 309/633] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:50:44 2022] PUT /identity/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/6763884b51204c488ca8c7e452e3a29e/roles/fd54d351b4ac49d7be6d0690309c00e7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:50:45.054612 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd37084-7e83-4212-a745-5ad8741c2137 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:45.054897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd37084-7e83-4212-a745-5ad8741c2137 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:45.055174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd37084-7e83-4212-a745-5ad8741c2137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:45.074555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cbd37084-7e83-4212-a745-5ad8741c2137 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:45.075815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cbd37084-7e83-4212-a745-5ad8741c2137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:45.092644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 325/634] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:45.098283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e0f0a-aaf8-40fd-93dc-bb4c883f87c0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:45.114946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:45.141071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:45.181664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 310/635] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:45.181664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa36e4-8c03-4442-951a-d5ab8972409d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:45.181664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa36e4-8c03-4442-951a-d5ab8972409d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:45.181664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa36e4-8c03-4442-951a-d5ab8972409d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:45.181664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 311/636] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:45.181664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa36e4-8c03-4442-951a-d5ab8972409d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:45.242601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e0f0a-aaf8-40fd-93dc-bb4c883f87c0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:45.280225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852ff8b-643a-4d48-aeda-085a755bc438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:45.281022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852ff8b-643a-4d48-aeda-085a755bc438 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:45.281302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852ff8b-643a-4d48-aeda-085a755bc438 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:45.281573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852ff8b-643a-4d48-aeda-085a755bc438 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:45.292420 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852ff8b-643a-4d48-aeda-085a755bc438 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:45.293920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852ff8b-643a-4d48-aeda-085a755bc438 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1852ff8b-643a-4d48-aeda-085a755bc438 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:45.313976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:45.315169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:45.316308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:45.316308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:45.316308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:45.316308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:45.316600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa36e4-8c03-4442-951a-d5ab8972409d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:45.320829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 326/637] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:45.322116 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7cf97-4ab2-48d9-a51e-330206ed285a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:45.322402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7cf97-4ab2-48d9-a51e-330206ed285a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:45.322675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7cf97-4ab2-48d9-a51e-330206ed285a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:45.339799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a9b7cf97-4ab2-48d9-a51e-330206ed285a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:45.341303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a9b7cf97-4ab2-48d9-a51e-330206ed285a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:45.357162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa31631-1130-470f-b625-f1dfe2edef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:45.358239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa31631-1130-470f-b625-f1dfe2edef76 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:45.358660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa31631-1130-470f-b625-f1dfe2edef76 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:45.358952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa31631-1130-470f-b625-f1dfe2edef76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:45.371178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa31631-1130-470f-b625-f1dfe2edef76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:45.372760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa31631-1130-470f-b625-f1dfe2edef76 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9fa31631-1130-470f-b625-f1dfe2edef76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:45.392952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:45.394118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:45.395262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:45.395262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:45.395262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:45.395262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:45.398952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 312/638] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:45.399870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa31631-1130-470f-b625-f1dfe2edef76 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:45.416215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:45.428159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 327/639] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:45.429182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7cf97-4ab2-48d9-a51e-330206ed285a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:45.444179 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:45.535566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa31631-1130-470f-b625-f1dfe2edef76 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:45.543383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7cf97-4ab2-48d9-a51e-330206ed285a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:45.579057 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6d202-b4a1-4595-bead-14ec3755cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:45.580188 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6d202-b4a1-4595-bead-14ec3755cf38 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:45.580467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6d202-b4a1-4595-bead-14ec3755cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:45.580924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6d202-b4a1-4595-bead-14ec3755cf38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:45.592060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d87121-7e86-4cfd-b5ae-2126a4b85326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:45.593012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d87121-7e86-4cfd-b5ae-2126a4b85326 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:45.593291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d87121-7e86-4cfd-b5ae-2126a4b85326 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:45.593568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d87121-7e86-4cfd-b5ae-2126a4b85326 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:45.594336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d87121-7e86-4cfd-b5ae-2126a4b85326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:45.594657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d87121-7e86-4cfd-b5ae-2126a4b85326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:45.596058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d87121-7e86-4cfd-b5ae-2126a4b85326 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:45.600402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6d202-b4a1-4595-bead-14ec3755cf38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:45.613763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 328/640] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:45.613763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d87121-7e86-4cfd-b5ae-2126a4b85326 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5ab6d202-b4a1-4595-bead-14ec3755cf38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:45.617409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:45.618537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:45.619441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:45.619441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:45.619441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:45.619441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:45.619441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:45.619441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:45.619441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:45.619441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:45.619441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:45.619441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:45.619441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:45.619441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:45.625673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 313/641] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:45.625673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6d202-b4a1-4595-bead-14ec3755cf38 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:45.706917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:45.717943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:45.745713 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d87121-7e86-4cfd-b5ae-2126a4b85326 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:45.747482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6d202-b4a1-4595-bead-14ec3755cf38 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:45.786976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85925b68-e84a-47ab-a1af-3afd11a6334f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:45.787669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85925b68-e84a-47ab-a1af-3afd11a6334f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:45.787808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d233b6-ff27-4aab-9fe7-733d2093c58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:45.788062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85925b68-e84a-47ab-a1af-3afd11a6334f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:45.788430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85925b68-e84a-47ab-a1af-3afd11a6334f None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:45.788791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d233b6-ff27-4aab-9fe7-733d2093c58c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:45.789289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d233b6-ff27-4aab-9fe7-733d2093c58c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:45.789407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85925b68-e84a-47ab-a1af-3afd11a6334f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:45.789575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d233b6-ff27-4aab-9fe7-733d2093c58c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:45.789759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85925b68-e84a-47ab-a1af-3afd11a6334f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:45.791326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85925b68-e84a-47ab-a1af-3afd11a6334f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:45.800735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d233b6-ff27-4aab-9fe7-733d2093c58c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:45.807274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 329/642] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:45.808354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85925b68-e84a-47ab-a1af-3afd11a6334f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a8d233b6-ff27-4aab-9fe7-733d2093c58c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:45.808723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:45.809785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:45.810673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 314/643] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:45.811402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d233b6-ff27-4aab-9fe7-733d2093c58c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:45.929970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85925b68-e84a-47ab-a1af-3afd11a6334f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:45.944157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d233b6-ff27-4aab-9fe7-733d2093c58c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:45.963022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:45.969713 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509ee56-c616-441d-95c0-6f42e3ef67e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:45.970589 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509ee56-c616-441d-95c0-6f42e3ef67e3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:45.970876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509ee56-c616-441d-95c0-6f42e3ef67e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:45.971156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509ee56-c616-441d-95c0-6f42e3ef67e3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:45.980499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509ee56-c616-441d-95c0-6f42e3ef67e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:45.981985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509ee56-c616-441d-95c0-6f42e3ef67e3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:45.984713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefaaacc-e253-4dc4-bb64-5f894538152e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:45.985441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaaacc-e253-4dc4-bb64-5f894538152e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:45.985781 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaaacc-e253-4dc4-bb64-5f894538152e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:45.986056 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaaacc-e253-4dc4-bb64-5f894538152e None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:45.986814 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefaaacc-e253-4dc4-bb64-5f894538152e None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:45.987130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefaaacc-e253-4dc4-bb64-5f894538152e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:45.988539 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefaaacc-e253-4dc4-bb64-5f894538152e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:45.990346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:45.998719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 315/644] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:45.999397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefaaacc-e253-4dc4-bb64-5f894538152e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9509ee56-c616-441d-95c0-6f42e3ef67e3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:46.004627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:46.006188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:46.007507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:46.007507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:46.007507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:46.007507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:46.007507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:46.007507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:46.007507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:46.007507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:46.007507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 07:50:46.007507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:46.007507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 330/645] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:46.011789 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509ee56-c616-441d-95c0-6f42e3ef67e3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:46.114663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefaaacc-e253-4dc4-bb64-5f894538152e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:46.134280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509ee56-c616-441d-95c0-6f42e3ef67e3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:46.155232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991c163-4e03-4e02-beba-0892bc38557b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:46.155904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991c163-4e03-4e02-beba-0892bc38557b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:46.156180 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991c163-4e03-4e02-beba-0892bc38557b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:46.156457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991c163-4e03-4e02-beba-0892bc38557b None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:46.157198 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991c163-4e03-4e02-beba-0892bc38557b None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:46.157503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991c163-4e03-4e02-beba-0892bc38557b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:46.158898 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991c163-4e03-4e02-beba-0892bc38557b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:46.167667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 316/646] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:46.168416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991c163-4e03-4e02-beba-0892bc38557b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:46.177204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7a71b-f8e9-4633-99a1-e00023b452af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:46.178081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7a71b-f8e9-4633-99a1-e00023b452af None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:46.178361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7a71b-f8e9-4633-99a1-e00023b452af None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:46.178692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7a71b-f8e9-4633-99a1-e00023b452af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:46.186727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7a71b-f8e9-4633-99a1-e00023b452af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:46.188204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7a71b-f8e9-4633-99a1-e00023b452af None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9fd7a71b-f8e9-4633-99a1-e00023b452af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:46.204694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:46.205733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:46.206680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:46.206680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:46.206680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:46.206680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:46.206680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:46.206680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:46.206680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:46.206680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:46.206680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 331/647] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:46.207177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7a71b-f8e9-4633-99a1-e00023b452af None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:46.243760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:46.263704 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:46.292467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991c163-4e03-4e02-beba-0892bc38557b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:46.318282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7a71b-f8e9-4633-99a1-e00023b452af None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:46.330605 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e4101-7096-468e-871b-1a3f70e6b92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:46.331502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e4101-7096-468e-871b-1a3f70e6b92c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:46.331868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e4101-7096-468e-871b-1a3f70e6b92c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:46.332211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e4101-7096-468e-871b-1a3f70e6b92c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:46.344562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e4101-7096-468e-871b-1a3f70e6b92c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:46.346389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e4101-7096-468e-871b-1a3f70e6b92c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:46.354675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1732f-d30a-4d91-9a9b-e11300cbbde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:46.355437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1732f-d30a-4d91-9a9b-e11300cbbde0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:46.355720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1732f-d30a-4d91-9a9b-e11300cbbde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:46.355991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1732f-d30a-4d91-9a9b-e11300cbbde0 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:46.356831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1732f-d30a-4d91-9a9b-e11300cbbde0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:46.357137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1732f-d30a-4d91-9a9b-e11300cbbde0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:46.358648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1732f-d30a-4d91-9a9b-e11300cbbde0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-918e4101-7096-468e-871b-1a3f70e6b92c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:46.365493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:46.366609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:46.367822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:46.367822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:46.367822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:46.367822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:46.372623 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 317/648] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:46.372623 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e4101-7096-468e-871b-1a3f70e6b92c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:46.384470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 332/649] 10.222.0.43 () {56 vars in 1137 bytes} [Mon Jul 4 07:50:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:46.385442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1732f-d30a-4d91-9a9b-e11300cbbde0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:46.483223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e4101-7096-468e-871b-1a3f70e6b92c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:46.495385 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:46.498370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1732f-d30a-4d91-9a9b-e11300cbbde0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:46.511314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:46.521521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6eb88-2e22-4bde-820d-8f94971ef29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:46.522533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6eb88-2e22-4bde-820d-8f94971ef29b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:46.522914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6eb88-2e22-4bde-820d-8f94971ef29b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:46.523311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6eb88-2e22-4bde-820d-8f94971ef29b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:46.535469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6eb88-2e22-4bde-820d-8f94971ef29b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:46.537082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6eb88-2e22-4bde-820d-8f94971ef29b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:46.540698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dd5de-9a22-49d1-9e43-1ffec99f25ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:46.541378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dd5de-9a22-49d1-9e43-1ffec99f25ee None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:46.541685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dd5de-9a22-49d1-9e43-1ffec99f25ee None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:46.541964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dd5de-9a22-49d1-9e43-1ffec99f25ee None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:46.542762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dd5de-9a22-49d1-9e43-1ffec99f25ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:46.543072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dd5de-9a22-49d1-9e43-1ffec99f25ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:46.544517 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dd5de-9a22-49d1-9e43-1ffec99f25ee None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-86d6eb88-2e22-4bde-820d-8f94971ef29b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:46.556317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:46.557282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:46.558569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:46.558569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:46.558569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:46.558569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:46.558810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 333/650] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:46.558810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dd5de-9a22-49d1-9e43-1ffec99f25ee None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:46.562030 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 318/651] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:46.563151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6eb88-2e22-4bde-820d-8f94971ef29b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:46.678481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6eb88-2e22-4bde-820d-8f94971ef29b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:46.680389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dd5de-9a22-49d1-9e43-1ffec99f25ee None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:46.713586 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6b92f-356d-4396-85ce-a95e47305d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:46.714259 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6b92f-356d-4396-85ce-a95e47305d50 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:46.714545 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6b92f-356d-4396-85ce-a95e47305d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:46.714822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6b92f-356d-4396-85ce-a95e47305d50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:46.716883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9561b4-3f84-4fd1-b22d-1f6649aefd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:46.717557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9561b4-3f84-4fd1-b22d-1f6649aefd3f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:46.717871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9561b4-3f84-4fd1-b22d-1f6649aefd3f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:46.718150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9561b4-3f84-4fd1-b22d-1f6649aefd3f None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:46.718883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9561b4-3f84-4fd1-b22d-1f6649aefd3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:46.719189 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9561b4-3f84-4fd1-b22d-1f6649aefd3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:46.720571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9561b4-3f84-4fd1-b22d-1f6649aefd3f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:46.734574 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 334/652] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:46.735430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9561b4-3f84-4fd1-b22d-1f6649aefd3f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:46.752295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:46.769400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:46.797234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6b92f-356d-4396-85ce-a95e47305d50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:46.798870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6b92f-356d-4396-85ce-a95e47305d50 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:46.850442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9561b4-3f84-4fd1-b22d-1f6649aefd3f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:46.890590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd6bfd-79be-4f1b-b3a6-284c16847950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:46.891251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd6bfd-79be-4f1b-b3a6-284c16847950 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:46.891537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd6bfd-79be-4f1b-b3a6-284c16847950 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:46.891814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd6bfd-79be-4f1b-b3a6-284c16847950 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:46.892657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd6bfd-79be-4f1b-b3a6-284c16847950 None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=15211e42ec9d4c1699d877f0b55ce918) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:46.892972 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd6bfd-79be-4f1b-b3a6-284c16847950 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:46.894488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd6bfd-79be-4f1b-b3a6-284c16847950 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:46.913667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 335/653] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:46 2022] GET /identity/v3/role_assignments?user.id=15211e42ec9d4c1699d877f0b55ce918&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:46.913667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd6bfd-79be-4f1b-b3a6-284c16847950 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:46.914124 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 319/654] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:46.914124 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6b92f-356d-4396-85ce-a95e47305d50 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:47.014379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:47.021292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:47.033389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd6bfd-79be-4f1b-b3a6-284c16847950 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:47.072944 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6b92f-356d-4396-85ce-a95e47305d50 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:47.080794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3e02b-7e50-4f95-8842-6eb0880db9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:47.081614 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3e02b-7e50-4f95-8842-6eb0880db9ea None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:47.082022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3e02b-7e50-4f95-8842-6eb0880db9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:47.082401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3e02b-7e50-4f95-8842-6eb0880db9ea None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:47.083337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3e02b-7e50-4f95-8842-6eb0880db9ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:47.083747 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3e02b-7e50-4f95-8842-6eb0880db9ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:47.085400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3e02b-7e50-4f95-8842-6eb0880db9ea None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:47.098707 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 336/655] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:47.099692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3e02b-7e50-4f95-8842-6eb0880db9ea None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:47.118076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb821d5b-a1db-4237-bc16-c4a02c8e298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:47.119649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb821d5b-a1db-4237-bc16-c4a02c8e298e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:47.119765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb821d5b-a1db-4237-bc16-c4a02c8e298e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:47.120049 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb821d5b-a1db-4237-bc16-c4a02c8e298e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:47.130899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb821d5b-a1db-4237-bc16-c4a02c8e298e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:47.132503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb821d5b-a1db-4237-bc16-c4a02c8e298e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-cb821d5b-a1db-4237-bc16-c4a02c8e298e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:47.148701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:47.149926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:47.151135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:47.151135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:47.151135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:47.151135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:47.151135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:47.151135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:47.151135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:47.151135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:47.151135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 07:50:47.151135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:47.151135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 320/656] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:47.151135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb821d5b-a1db-4237-bc16-c4a02c8e298e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:47.224708 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3e02b-7e50-4f95-8842-6eb0880db9ea None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:47.262851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3df95d-9819-4b6b-9db0-7cc9d2d7da5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:47.263930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3df95d-9819-4b6b-9db0-7cc9d2d7da5f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:47.264554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3df95d-9819-4b6b-9db0-7cc9d2d7da5f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:47.264554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3df95d-9819-4b6b-9db0-7cc9d2d7da5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:47.265373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb821d5b-a1db-4237-bc16-c4a02c8e298e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:47.276108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3df95d-9819-4b6b-9db0-7cc9d2d7da5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:47.277903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3df95d-9819-4b6b-9db0-7cc9d2d7da5f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:47.291533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9e3df95d-9819-4b6b-9db0-7cc9d2d7da5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:47.296701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:47.297813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:47.298917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:47.298917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:47.298917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:47.298917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:47.304515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 337/657] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:47.304515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4b647-1d5d-415f-bcb8-414026adb468 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:47.304515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4b647-1d5d-415f-bcb8-414026adb468 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:47.304515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4b647-1d5d-415f-bcb8-414026adb468 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:47.304515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 338/658] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:47.304515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4b647-1d5d-415f-bcb8-414026adb468 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:47.308288 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:47.314753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15524618-2857-49d1-b453-b31b0dc95d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:47.315592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15524618-2857-49d1-b453-b31b0dc95d28 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:47.315900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15524618-2857-49d1-b453-b31b0dc95d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:47.316181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15524618-2857-49d1-b453-b31b0dc95d28 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:47.324508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15524618-2857-49d1-b453-b31b0dc95d28 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:47.326111 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15524618-2857-49d1-b453-b31b0dc95d28 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-15524618-2857-49d1-b453-b31b0dc95d28 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:47.341673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:47.342900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:47.344072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:47.344072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:47.344072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:47.344072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:47.344072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:47.344072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:47.344072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:47.344072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:47.344072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 07:50:47.344072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:47.348087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 321/659] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:47.348964 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15524618-2857-49d1-b453-b31b0dc95d28 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:47.422108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4b647-1d5d-415f-bcb8-414026adb468 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:47.458036 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15524618-2857-49d1-b453-b31b0dc95d28 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:47.459783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01b40b-bed7-46ec-9c4c-dea897da1876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:47.460613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01b40b-bed7-46ec-9c4c-dea897da1876 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:47.460903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01b40b-bed7-46ec-9c4c-dea897da1876 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:47.461210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01b40b-bed7-46ec-9c4c-dea897da1876 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:47.462082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01b40b-bed7-46ec-9c4c-dea897da1876 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:47.462397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01b40b-bed7-46ec-9c4c-dea897da1876 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:47.463875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01b40b-bed7-46ec-9c4c-dea897da1876 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:47.483669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 339/660] 10.222.0.43 () {56 vars in 1137 bytes} [Mon Jul 4 07:50:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:47.484964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca7841-b18b-4f5b-b69e-072db9379c08 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:47.485252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca7841-b18b-4f5b-b69e-072db9379c08 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:47.485534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca7841-b18b-4f5b-b69e-072db9379c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:47.500929 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4274b6-24a9-4123-8fc3-b764e6832cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:47.501961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4274b6-24a9-4123-8fc3-b764e6832cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:47.502388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4274b6-24a9-4123-8fc3-b764e6832cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:47.502805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4274b6-24a9-4123-8fc3-b764e6832cd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:47.503275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5fca7841-b18b-4f5b-b69e-072db9379c08 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:47.503832 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4274b6-24a9-4123-8fc3-b764e6832cd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:47.504621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4274b6-24a9-4123-8fc3-b764e6832cd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:47.504831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5fca7841-b18b-4f5b-b69e-072db9379c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:47.506312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4274b6-24a9-4123-8fc3-b764e6832cd1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:47.519591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 322/661] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:47.520644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4274b6-24a9-4123-8fc3-b764e6832cd1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:47.567731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:47.573237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:47.589531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 340/662] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:47.589531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca7841-b18b-4f5b-b69e-072db9379c08 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:47.651441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4274b6-24a9-4123-8fc3-b764e6832cd1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:47.686546 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a489afc-8b2c-493e-92d1-dee4c61eaf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:47.687234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a489afc-8b2c-493e-92d1-dee4c61eaf9d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:47.687519 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a489afc-8b2c-493e-92d1-dee4c61eaf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:47.687796 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a489afc-8b2c-493e-92d1-dee4c61eaf9d None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:47.688639 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a489afc-8b2c-493e-92d1-dee4c61eaf9d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:47.688951 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a489afc-8b2c-493e-92d1-dee4c61eaf9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:47.690490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a489afc-8b2c-493e-92d1-dee4c61eaf9d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:47.691761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca7841-b18b-4f5b-b69e-072db9379c08 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:47.710417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 323/663] 10.222.0.43 () {56 vars in 1139 bytes} [Mon Jul 4 07:50:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:47.711597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a489afc-8b2c-493e-92d1-dee4c61eaf9d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:47.739251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f46d2-2050-45e4-9844-8ab09e5c0b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:47.739921 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f46d2-2050-45e4-9844-8ab09e5c0b7b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:47.740203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f46d2-2050-45e4-9844-8ab09e5c0b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:47.740477 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f46d2-2050-45e4-9844-8ab09e5c0b7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:47.802356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:47.813927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f46d2-2050-45e4-9844-8ab09e5c0b7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:47.815411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f46d2-2050-45e4-9844-8ab09e5c0b7b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:47.820389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:47.824038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a489afc-8b2c-493e-92d1-dee4c61eaf9d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:47.859699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1a82f-31be-4211-841f-5fead7a34b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:47.860766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1a82f-31be-4211-841f-5fead7a34b25 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:47.861053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1a82f-31be-4211-841f-5fead7a34b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:47.861327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1a82f-31be-4211-841f-5fead7a34b25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:47.868618 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1a82f-31be-4211-841f-5fead7a34b25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:47.870110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1a82f-31be-4211-841f-5fead7a34b25 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0ae1a82f-31be-4211-841f-5fead7a34b25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:47.885001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:47.886206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:47.887161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:47.887161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:47.887161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:47.887161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:47.887161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:47.887161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:47.887161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:47.887161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:47.887161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 324/664] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:47.887681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1a82f-31be-4211-841f-5fead7a34b25 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:47.933910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 341/665] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:47.933910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f46d2-2050-45e4-9844-8ab09e5c0b7b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:48.013403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1a82f-31be-4211-841f-5fead7a34b25 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:48.040892 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:48.052296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f46d2-2050-45e4-9844-8ab09e5c0b7b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:48.060789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b3336-4123-4b19-8000-909097edf323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:48.061923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b3336-4123-4b19-8000-909097edf323 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:48.062402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b3336-4123-4b19-8000-909097edf323 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:48.062402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b3336-4123-4b19-8000-909097edf323 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:48.074407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b3336-4123-4b19-8000-909097edf323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-374b3336-4123-4b19-8000-909097edf323 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:48.082428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:48.083496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:48.084395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 325/666] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:48.085145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b3336-4123-4b19-8000-909097edf323 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:48.097501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df497a4-6c8b-4160-8b07-7175d7ab057d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:48.098203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df497a4-6c8b-4160-8b07-7175d7ab057d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:48.098487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df497a4-6c8b-4160-8b07-7175d7ab057d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:48.098753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df497a4-6c8b-4160-8b07-7175d7ab057d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:48.119702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:48.174647 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df497a4-6c8b-4160-8b07-7175d7ab057d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:48.176153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df497a4-6c8b-4160-8b07-7175d7ab057d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:48.229989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b3336-4123-4b19-8000-909097edf323 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:48.267485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a152b-6c8e-453b-b6f0-031a57514ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:48.268159 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a152b-6c8e-453b-b6f0-031a57514ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:48.268442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a152b-6c8e-453b-b6f0-031a57514ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:48.268719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a152b-6c8e-453b-b6f0-031a57514ff9 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:48.270009 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a152b-6c8e-453b-b6f0-031a57514ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:48.270325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a152b-6c8e-453b-b6f0-031a57514ff9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:48.272147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a152b-6c8e-453b-b6f0-031a57514ff9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:48.288158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:48.290289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 326/667] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:48.291229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a152b-6c8e-453b-b6f0-031a57514ff9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:48.309697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 342/668] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:48.309697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df497a4-6c8b-4160-8b07-7175d7ab057d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:48.403454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:48.424633 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a152b-6c8e-453b-b6f0-031a57514ff9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:48.428932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df497a4-6c8b-4160-8b07-7175d7ab057d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:48.463922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1e053-2f07-420a-b448-4bfd2de95fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:48.464663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1e053-2f07-420a-b448-4bfd2de95fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:48.464988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1e053-2f07-420a-b448-4bfd2de95fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:48.465361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1e053-2f07-420a-b448-4bfd2de95fa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:48.466984 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215672ea-0d68-4157-8215-490008c34163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:48.467623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215672ea-0d68-4157-8215-490008c34163 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:48.467909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215672ea-0d68-4157-8215-490008c34163 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:48.468184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215672ea-0d68-4157-8215-490008c34163 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:48.468941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215672ea-0d68-4157-8215-490008c34163 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:48.469248 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215672ea-0d68-4157-8215-490008c34163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:48.470698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215672ea-0d68-4157-8215-490008c34163 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:48.476951 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1e053-2f07-420a-b448-4bfd2de95fa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:48.478442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1e053-2f07-420a-b448-4bfd2de95fa2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:48.484425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 343/669] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:48.484425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215672ea-0d68-4157-8215-490008c34163 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5ba1e053-2f07-420a-b448-4bfd2de95fa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:48.507204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:48.508211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:48.509138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:48.509138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:48.509138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:48.509138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:48.509138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 327/670] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:48.509433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1e053-2f07-420a-b448-4bfd2de95fa2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:48.549430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:48.601869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215672ea-0d68-4157-8215-490008c34163 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:48.627517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1e053-2f07-420a-b448-4bfd2de95fa2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:48.638172 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9d799-a3c3-4751-905e-ea4a70b8b308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:48.638838 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9d799-a3c3-4751-905e-ea4a70b8b308 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:48.639141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9d799-a3c3-4751-905e-ea4a70b8b308 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:48.639388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9d799-a3c3-4751-905e-ea4a70b8b308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:48.640225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9d799-a3c3-4751-905e-ea4a70b8b308 None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=061ffcf242ff4155b92daa8c3888fd10) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:48.640538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9d799-a3c3-4751-905e-ea4a70b8b308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:48.642018 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9d799-a3c3-4751-905e-ea4a70b8b308 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:48.658453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 344/671] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:48 2022] GET /identity/v3/role_assignments?user.id=061ffcf242ff4155b92daa8c3888fd10&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:48.659519 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9d799-a3c3-4751-905e-ea4a70b8b308 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:48.668388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:48.682225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1185a1-5deb-40fb-a7ed-eef49b0bfbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:48.683054 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1185a1-5deb-40fb-a7ed-eef49b0bfbfa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:48.683339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1185a1-5deb-40fb-a7ed-eef49b0bfbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:48.683601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1185a1-5deb-40fb-a7ed-eef49b0bfbfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:48.699729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1185a1-5deb-40fb-a7ed-eef49b0bfbfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:48.701249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1185a1-5deb-40fb-a7ed-eef49b0bfbfa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7f1185a1-5deb-40fb-a7ed-eef49b0bfbfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:48.722665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:48.724508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:48.726748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:48.726748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:48.726748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:48.726748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:48.729997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 328/672] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:48.731127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1185a1-5deb-40fb-a7ed-eef49b0bfbfa None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:48.840364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9d799-a3c3-4751-905e-ea4a70b8b308 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:48.862738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:48.879422 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1185a1-5deb-40fb-a7ed-eef49b0bfbfa None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:48.894385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95521a-8246-46f3-a415-3e92a00c1be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:48.895257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95521a-8246-46f3-a415-3e92a00c1be2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:48.895545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95521a-8246-46f3-a415-3e92a00c1be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:48.895823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95521a-8246-46f3-a415-3e92a00c1be2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:48.911997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95521a-8246-46f3-a415-3e92a00c1be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:48.914399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95521a-8246-46f3-a415-3e92a00c1be2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:48.922284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43667008-775a-4f1b-87a9-b527ef8be14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:48.923037 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43667008-775a-4f1b-87a9-b527ef8be14e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:48.923501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43667008-775a-4f1b-87a9-b527ef8be14e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:48.923908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43667008-775a-4f1b-87a9-b527ef8be14e None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:48.925181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43667008-775a-4f1b-87a9-b527ef8be14e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:48.925718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43667008-775a-4f1b-87a9-b527ef8be14e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:48.928027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43667008-775a-4f1b-87a9-b527ef8be14e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5a95521a-8246-46f3-a415-3e92a00c1be2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:48.934198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:48.935183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:48.936502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:48.936502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:48.936502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:48.936502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:48.936502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 345/673] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:48.936790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95521a-8246-46f3-a415-3e92a00c1be2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:48.950023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 329/674] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:48.950913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43667008-775a-4f1b-87a9-b527ef8be14e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:49.013792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:49.058754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95521a-8246-46f3-a415-3e92a00c1be2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:49.080405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43667008-775a-4f1b-87a9-b527ef8be14e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:49.098531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d0182-f5d9-4c74-bfa9-5c1acc185375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:49.099214 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d0182-f5d9-4c74-bfa9-5c1acc185375 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:49.099528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d0182-f5d9-4c74-bfa9-5c1acc185375 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:49.099812 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d0182-f5d9-4c74-bfa9-5c1acc185375 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:49.100572 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d0182-f5d9-4c74-bfa9-5c1acc185375 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:49.101521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d0182-f5d9-4c74-bfa9-5c1acc185375 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:49.105853 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d0182-f5d9-4c74-bfa9-5c1acc185375 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:49.118678 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854b071-237d-4986-8c03-6b6d5b779bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:49.119018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854b071-237d-4986-8c03-6b6d5b779bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:49.119204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854b071-237d-4986-8c03-6b6d5b779bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:49.119471 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854b071-237d-4986-8c03-6b6d5b779bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:49.120206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854b071-237d-4986-8c03-6b6d5b779bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:49.120518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854b071-237d-4986-8c03-6b6d5b779bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:49.121938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854b071-237d-4986-8c03-6b6d5b779bd9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:49.130726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 346/675] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:49.130726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d0182-f5d9-4c74-bfa9-5c1acc185375 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:49.137114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 330/676] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:49.141661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca44b69-a5b3-40b8-bdcb-930834cf9043 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:49.141661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca44b69-a5b3-40b8-bdcb-930834cf9043 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:49.141661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca44b69-a5b3-40b8-bdcb-930834cf9043 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:49.141661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 331/677] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:49.141661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca44b69-a5b3-40b8-bdcb-930834cf9043 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:49.168156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:49.258399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d0182-f5d9-4c74-bfa9-5c1acc185375 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:49.259906 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca44b69-a5b3-40b8-bdcb-930834cf9043 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:49.264179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:49.296962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c17584-ea81-408d-b940-09552671dfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:49.298004 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c17584-ea81-408d-b940-09552671dfe8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:49.298361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c17584-ea81-408d-b940-09552671dfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:49.298714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c17584-ea81-408d-b940-09552671dfe8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:49.300202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845ab5a-e264-4e6e-85ef-16302f683157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:49.301121 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845ab5a-e264-4e6e-85ef-16302f683157 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:49.301404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845ab5a-e264-4e6e-85ef-16302f683157 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:49.302008 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845ab5a-e264-4e6e-85ef-16302f683157 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:49.307401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c17584-ea81-408d-b940-09552671dfe8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:49.308899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c17584-ea81-408d-b940-09552671dfe8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:49.309375 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845ab5a-e264-4e6e-85ef-16302f683157 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:49.310853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845ab5a-e264-4e6e-85ef-16302f683157 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-52c17584-ea81-408d-b940-09552671dfe8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:49.323858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:49.324798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:49.330814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:49.330814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:49.330814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:49.330814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:49.330814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:49.330814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:49.330814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:49.330814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1845ab5a-e264-4e6e-85ef-16302f683157 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:49.331257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:49.332090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:49.333010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:49.333010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:49.333010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:49.333010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:49.333010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:49.333010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:49.333010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:49.333010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:49.333456 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 347/678] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:49.333527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 332/679] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:49.333527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae37d0-edd0-4ab8-8e74-64be6ac1d83f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:49.333527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae37d0-edd0-4ab8-8e74-64be6ac1d83f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:49.333527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae37d0-edd0-4ab8-8e74-64be6ac1d83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:49.334777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c17584-ea81-408d-b940-09552671dfe8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:49.350201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6fae37d0-edd0-4ab8-8e74-64be6ac1d83f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:49.351591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6fae37d0-edd0-4ab8-8e74-64be6ac1d83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:49.432860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 333/680] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:49.433723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae37d0-edd0-4ab8-8e74-64be6ac1d83f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:49.438234 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:49.450421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c17584-ea81-408d-b940-09552671dfe8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:49.489246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf602c7-8c2a-408c-bf2f-e14bf7918e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:49.490299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf602c7-8c2a-408c-bf2f-e14bf7918e2a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:49.490663 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf602c7-8c2a-408c-bf2f-e14bf7918e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:49.491041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf602c7-8c2a-408c-bf2f-e14bf7918e2a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:49.500992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf602c7-8c2a-408c-bf2f-e14bf7918e2a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:49.502681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf602c7-8c2a-408c-bf2f-e14bf7918e2a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ddf602c7-8c2a-408c-bf2f-e14bf7918e2a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:49.518208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:49.519459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:49.520697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:49.520697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:49.520697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:49.520697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:49.520697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:49.520697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:49.520697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:49.520697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:49.520697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 07:50:49.520697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:49.521426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 348/681] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:50:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:49.522205 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf602c7-8c2a-408c-bf2f-e14bf7918e2a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:49.557222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:49.564900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae37d0-edd0-4ab8-8e74-64be6ac1d83f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:49.601209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54117882-a0df-4edf-b1d4-f94f5e60580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:49.601996 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54117882-a0df-4edf-b1d4-f94f5e60580b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:49.602345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54117882-a0df-4edf-b1d4-f94f5e60580b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:49.602688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54117882-a0df-4edf-b1d4-f94f5e60580b None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:49.603544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54117882-a0df-4edf-b1d4-f94f5e60580b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:49.603917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54117882-a0df-4edf-b1d4-f94f5e60580b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:49.605445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54117882-a0df-4edf-b1d4-f94f5e60580b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:49.618003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 334/682] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:49.618824 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54117882-a0df-4edf-b1d4-f94f5e60580b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:49.642871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf602c7-8c2a-408c-bf2f-e14bf7918e2a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:49.680893 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747b896-48e2-4b21-9ee3-775bfafc194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:49.681567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747b896-48e2-4b21-9ee3-775bfafc194c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:49.681884 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747b896-48e2-4b21-9ee3-775bfafc194c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:49.682165 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747b896-48e2-4b21-9ee3-775bfafc194c None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:49.683017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747b896-48e2-4b21-9ee3-775bfafc194c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:49.683272 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747b896-48e2-4b21-9ee3-775bfafc194c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:49.684721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747b896-48e2-4b21-9ee3-775bfafc194c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:49.697853 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 349/683] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:49.698684 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747b896-48e2-4b21-9ee3-775bfafc194c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:49.712703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:49.731429 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54117882-a0df-4edf-b1d4-f94f5e60580b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:49.770080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27010dfd-3d70-498b-a124-4fda7b369f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:49.770891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27010dfd-3d70-498b-a124-4fda7b369f62 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:49.771175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27010dfd-3d70-498b-a124-4fda7b369f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:49.771454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27010dfd-3d70-498b-a124-4fda7b369f62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:49.784971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27010dfd-3d70-498b-a124-4fda7b369f62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-27010dfd-3d70-498b-a124-4fda7b369f62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:49.796619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:49.797794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:49.798833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:49.798833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:49.798833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:49.798833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:49.798833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:49.798833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:49.798833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:49.798833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:49.798833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:49.798833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:49.798833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:50:49.798833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:49.803392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 335/684] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:50:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:49.805251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27010dfd-3d70-498b-a124-4fda7b369f62 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:49.814198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747b896-48e2-4b21-9ee3-775bfafc194c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:49.829416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:49.852655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1569bf-3116-46b1-9823-7f4a7a3eca36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:49.853336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1569bf-3116-46b1-9823-7f4a7a3eca36 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:49.853616 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1569bf-3116-46b1-9823-7f4a7a3eca36 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:49.853915 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1569bf-3116-46b1-9823-7f4a7a3eca36 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:49.854821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1569bf-3116-46b1-9823-7f4a7a3eca36 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:49.855129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1569bf-3116-46b1-9823-7f4a7a3eca36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:49.856784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1569bf-3116-46b1-9823-7f4a7a3eca36 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:49.879038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 350/685] 10.222.0.43 () {56 vars in 1135 bytes} [Mon Jul 4 07:50:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:49.880265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1569bf-3116-46b1-9823-7f4a7a3eca36 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:49.931429 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27010dfd-3d70-498b-a124-4fda7b369f62 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:49.958104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:49.969684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493281e-350c-4b02-a8ba-dfbe9f156335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:49.970342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493281e-350c-4b02-a8ba-dfbe9f156335 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:49.970627 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493281e-350c-4b02-a8ba-dfbe9f156335 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:49.970897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493281e-350c-4b02-a8ba-dfbe9f156335 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:49.971724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493281e-350c-4b02-a8ba-dfbe9f156335 None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=4a122422a45446c9a63d67b4f931fa47) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:49.972032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493281e-350c-4b02-a8ba-dfbe9f156335 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:49.973463 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493281e-350c-4b02-a8ba-dfbe9f156335 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:49.979333 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 336/686] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:49 2022] GET /identity/v3/role_assignments?user.id=4a122422a45446c9a63d67b4f931fa47&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:49.980109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493281e-350c-4b02-a8ba-dfbe9f156335 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:50.015822 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1569bf-3116-46b1-9823-7f4a7a3eca36 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:50.070354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6cce7-1bb2-4bcf-b088-529b2dad0e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:50.071085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6cce7-1bb2-4bcf-b088-529b2dad0e0c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:50.071431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6cce7-1bb2-4bcf-b088-529b2dad0e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:50.072938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6cce7-1bb2-4bcf-b088-529b2dad0e0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:50.073862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6cce7-1bb2-4bcf-b088-529b2dad0e0c None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=429e173890b84ee289abc9793794ddb9) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:50.074182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6cce7-1bb2-4bcf-b088-529b2dad0e0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:50.075660 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6cce7-1bb2-4bcf-b088-529b2dad0e0c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:50.083221 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:50.085365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 351/687] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:49 2022] GET /identity/v3/role_assignments?user.id=429e173890b84ee289abc9793794ddb9&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:50.087658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6cce7-1bb2-4bcf-b088-529b2dad0e0c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:50.168191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493281e-350c-4b02-a8ba-dfbe9f156335 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:50.222511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0e29f-76d0-4009-8cd6-8bcd51742c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:50.223981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0e29f-76d0-4009-8cd6-8bcd51742c38 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:50.223981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0e29f-76d0-4009-8cd6-8bcd51742c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:50.223981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0e29f-76d0-4009-8cd6-8bcd51742c38 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:50.224393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0e29f-76d0-4009-8cd6-8bcd51742c38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:50.224695 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0e29f-76d0-4009-8cd6-8bcd51742c38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:50.226360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0e29f-76d0-4009-8cd6-8bcd51742c38 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:50.235487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6cce7-1bb2-4bcf-b088-529b2dad0e0c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:50.236273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 337/688] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:50.241719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0e29f-76d0-4009-8cd6-8bcd51742c38 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:50.272285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:50.277654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefaf37-43ed-424b-93e4-9e258f8b111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:50.278367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefaf37-43ed-424b-93e4-9e258f8b111a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:50.278645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefaf37-43ed-424b-93e4-9e258f8b111a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:50.278918 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefaf37-43ed-424b-93e4-9e258f8b111a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:50.365957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0e29f-76d0-4009-8cd6-8bcd51742c38 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:50.372268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefaf37-43ed-424b-93e4-9e258f8b111a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:50.374279 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefaf37-43ed-424b-93e4-9e258f8b111a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:50.410347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59987860-6112-4354-a1bd-e01007d18b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:50.411697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59987860-6112-4354-a1bd-e01007d18b22 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:50.412169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59987860-6112-4354-a1bd-e01007d18b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:50.412629 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59987860-6112-4354-a1bd-e01007d18b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:50.429720 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:50.431907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59987860-6112-4354-a1bd-e01007d18b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:50.433854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59987860-6112-4354-a1bd-e01007d18b22 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-59987860-6112-4354-a1bd-e01007d18b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:50.455132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:50.456776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:50.458186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:50.458186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:50.458186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:50.458186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:50.459323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 338/689] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:50.464470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59987860-6112-4354-a1bd-e01007d18b22 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:50.578194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 352/690] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:50.583566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefaf37-43ed-424b-93e4-9e258f8b111a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:50.644576 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:50.700277 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59987860-6112-4354-a1bd-e01007d18b22 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:50.767763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f63d2f-b701-4c43-b1d1-09e358510e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:50.768773 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f63d2f-b701-4c43-b1d1-09e358510e31 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:50.770372 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f63d2f-b701-4c43-b1d1-09e358510e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:50.770798 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f63d2f-b701-4c43-b1d1-09e358510e31 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:50.771993 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f63d2f-b701-4c43-b1d1-09e358510e31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:50.774620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f63d2f-b701-4c43-b1d1-09e358510e31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:50.776493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f63d2f-b701-4c43-b1d1-09e358510e31 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:50.794812 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 339/691] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:50.798102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefaf37-43ed-424b-93e4-9e258f8b111a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:50.802667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f63d2f-b701-4c43-b1d1-09e358510e31 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:50.837834 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e710a-dda5-4cfd-938f-be5adbe4b2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:50.839047 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e710a-dda5-4cfd-938f-be5adbe4b2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:50.839539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e710a-dda5-4cfd-938f-be5adbe4b2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:50.840066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e710a-dda5-4cfd-938f-be5adbe4b2a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:50.852304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e710a-dda5-4cfd-938f-be5adbe4b2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:50.854661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e710a-dda5-4cfd-938f-be5adbe4b2a4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d90e710a-dda5-4cfd-938f-be5adbe4b2a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:50.874365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:50.875327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:50.876311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:50.876311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:50.876311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:50.876311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:50.878668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 353/692] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:50.890675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e710a-dda5-4cfd-938f-be5adbe4b2a4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:50.916047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:50.984718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f63d2f-b701-4c43-b1d1-09e358510e31 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:51.004071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:51.026809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e710a-dda5-4cfd-938f-be5adbe4b2a4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:51.032799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85ce80-14b6-4c1f-b0da-7d6ea52a3b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:51.033793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85ce80-14b6-4c1f-b0da-7d6ea52a3b9e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:51.034184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85ce80-14b6-4c1f-b0da-7d6ea52a3b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:51.034564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85ce80-14b6-4c1f-b0da-7d6ea52a3b9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:51.043709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85ce80-14b6-4c1f-b0da-7d6ea52a3b9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:51.045480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85ce80-14b6-4c1f-b0da-7d6ea52a3b9e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ca85ce80-14b6-4c1f-b0da-7d6ea52a3b9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:51.061698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:51.062973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:51.064155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:51.064155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:51.064155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:51.064155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:51.064155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:51.064155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:51.064155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:51.064155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:51.064155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:50:51.064155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:51.066152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 340/693] 10.222.0.43 () {56 vars in 1093 bytes} [Mon Jul 4 07:50:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:51.067752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fd1e4-a454-42ec-bb3c-ba1b711d2abe None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:51.068146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fd1e4-a454-42ec-bb3c-ba1b711d2abe None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:51.068521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fd1e4-a454-42ec-bb3c-ba1b711d2abe None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:51.070615 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 341/694] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:51.074586 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fd1e4-a454-42ec-bb3c-ba1b711d2abe None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:51.081084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0538afb-d5d8-4aff-b9fb-941464d6c4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:51.082091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0538afb-d5d8-4aff-b9fb-941464d6c4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:51.082623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0538afb-d5d8-4aff-b9fb-941464d6c4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:51.083139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0538afb-d5d8-4aff-b9fb-941464d6c4d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:51.084044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0538afb-d5d8-4aff-b9fb-941464d6c4d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:51.084495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0538afb-d5d8-4aff-b9fb-941464d6c4d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:51.086216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0538afb-d5d8-4aff-b9fb-941464d6c4d7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:51.104347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 354/695] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:51.106268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820412a-40cf-4431-8bb7-79dd7451409f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:51.106726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820412a-40cf-4431-8bb7-79dd7451409f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:51.107126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820412a-40cf-4431-8bb7-79dd7451409f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:51.133999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b820412a-40cf-4431-8bb7-79dd7451409f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:51.135550 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b820412a-40cf-4431-8bb7-79dd7451409f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:51.213777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 355/696] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:51.214927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820412a-40cf-4431-8bb7-79dd7451409f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:51.218924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fd1e4-a454-42ec-bb3c-ba1b711d2abe None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:51.243319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:51.269199 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f109f5-4d4a-4985-90c9-92a3fd9a266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:51.270358 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f109f5-4d4a-4985-90c9-92a3fd9a266c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:51.270823 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f109f5-4d4a-4985-90c9-92a3fd9a266c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:51.271269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f109f5-4d4a-4985-90c9-92a3fd9a266c None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:51.272667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f109f5-4d4a-4985-90c9-92a3fd9a266c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:51.273162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f109f5-4d4a-4985-90c9-92a3fd9a266c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:51.274839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f109f5-4d4a-4985-90c9-92a3fd9a266c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:51.293907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 342/697] 10.222.0.43 () {56 vars in 1133 bytes} [Mon Jul 4 07:50:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:50:51.295045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f109f5-4d4a-4985-90c9-92a3fd9a266c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:51.315140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:51.344263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820412a-40cf-4431-8bb7-79dd7451409f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:51.384067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30a153-2859-4f73-979b-a11dffbb1c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:51.384936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30a153-2859-4f73-979b-a11dffbb1c96 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:51.385133 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30a153-2859-4f73-979b-a11dffbb1c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:51.385769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30a153-2859-4f73-979b-a11dffbb1c96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:51.393912 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30a153-2859-4f73-979b-a11dffbb1c96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:51.395554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30a153-2859-4f73-979b-a11dffbb1c96 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7d30a153-2859-4f73-979b-a11dffbb1c96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:51.411674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:51.412687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:51.413678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:51.413678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:51.413678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:51.413678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:51.413678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:51.413678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:51.413678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:51.413678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:51.415427 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f109f5-4d4a-4985-90c9-92a3fd9a266c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:51.418393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 356/698] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:51.419665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30a153-2859-4f73-979b-a11dffbb1c96 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:51.455339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c510894-63d0-477f-9eec-084307d165b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:51.456558 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c510894-63d0-477f-9eec-084307d165b8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:51.457035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c510894-63d0-477f-9eec-084307d165b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:51.457500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c510894-63d0-477f-9eec-084307d165b8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:51.458383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c510894-63d0-477f-9eec-084307d165b8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:51.460174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c510894-63d0-477f-9eec-084307d165b8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-3c510894-63d0-477f-9eec-084307d165b8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:51.477534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:51.478757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 07:50:51.479776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:51.482796 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 343/699] 10.222.0.43 () {56 vars in 1129 bytes} [Mon Jul 4 07:50:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:51.483857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c510894-63d0-477f-9eec-084307d165b8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:51.522770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:51.540057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30a153-2859-4f73-979b-a11dffbb1c96 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:51.578002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca82c41-cb61-4b86-856b-124551bdcecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:51.578644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca82c41-cb61-4b86-856b-124551bdcecc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:51.578935 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca82c41-cb61-4b86-856b-124551bdcecc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:51.579208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca82c41-cb61-4b86-856b-124551bdcecc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:51.582069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:51.604081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c510894-63d0-477f-9eec-084307d165b8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:51.643080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6454e02-f9bb-450b-b0f1-1cd2d2bcf58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:51.644537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6454e02-f9bb-450b-b0f1-1cd2d2bcf58a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:51.645030 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6454e02-f9bb-450b-b0f1-1cd2d2bcf58a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:51.645514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6454e02-f9bb-450b-b0f1-1cd2d2bcf58a None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:51.646521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6454e02-f9bb-450b-b0f1-1cd2d2bcf58a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:51.647185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6454e02-f9bb-450b-b0f1-1cd2d2bcf58a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:51.655866 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6454e02-f9bb-450b-b0f1-1cd2d2bcf58a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:51.660175 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca82c41-cb61-4b86-856b-124551bdcecc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:51.661679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca82c41-cb61-4b86-856b-124551bdcecc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:51.670394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 344/700] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:51.671578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6454e02-f9bb-450b-b0f1-1cd2d2bcf58a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:51.770929 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:51.778771 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6454e02-f9bb-450b-b0f1-1cd2d2bcf58a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:51.784786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 357/701] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 07:50:51.785753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca82c41-cb61-4b86-856b-124551bdcecc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:51.815740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba58882-9133-4ddf-8123-5fb78cb57a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:51.816669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba58882-9133-4ddf-8123-5fb78cb57a02 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:51.817091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba58882-9133-4ddf-8123-5fb78cb57a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:51.817532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba58882-9133-4ddf-8123-5fb78cb57a02 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:51.818439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba58882-9133-4ddf-8123-5fb78cb57a02 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:51.818886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba58882-9133-4ddf-8123-5fb78cb57a02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:51.820583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba58882-9133-4ddf-8123-5fb78cb57a02 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:51.824951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:51.830310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 345/702] 10.222.0.43 () {56 vars in 1124 bytes} [Mon Jul 4 07:50:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:51.831523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba58882-9133-4ddf-8123-5fb78cb57a02 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:51.904741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca82c41-cb61-4b86-856b-124551bdcecc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:51.938874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba58882-9133-4ddf-8123-5fb78cb57a02 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:51.943564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fae40f-607a-4dc2-ae0d-dd95b0256d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:51.944268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fae40f-607a-4dc2-ae0d-dd95b0256d6b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:51.944714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fae40f-607a-4dc2-ae0d-dd95b0256d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:51.945214 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fae40f-607a-4dc2-ae0d-dd95b0256d6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:51.946178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fae40f-607a-4dc2-ae0d-dd95b0256d6b None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=6763884b51204c488ca8c7e452e3a29e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:51.946504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fae40f-607a-4dc2-ae0d-dd95b0256d6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:51.947994 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fae40f-607a-4dc2-ae0d-dd95b0256d6b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:51.961973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 358/703] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:51 2022] GET /identity/v3/role_assignments?user.id=6763884b51204c488ca8c7e452e3a29e&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:51.963130 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fae40f-607a-4dc2-ae0d-dd95b0256d6b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:51.985327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd9dd7-c236-46eb-8e56-c4900c8b5298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:51.986845 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd9dd7-c236-46eb-8e56-c4900c8b5298 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:51.988120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd9dd7-c236-46eb-8e56-c4900c8b5298 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:51.989448 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd9dd7-c236-46eb-8e56-c4900c8b5298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:52.014313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd9dd7-c236-46eb-8e56-c4900c8b5298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:52.016231 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd9dd7-c236-46eb-8e56-c4900c8b5298 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-00fd9dd7-c236-46eb-8e56-c4900c8b5298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:52.057165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:52.058416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:52.059665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:52.059665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:52.059665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:52.059665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:52.059665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 346/704] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:52.064430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd9dd7-c236-46eb-8e56-c4900c8b5298 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:52.084115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:52.187973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:52.195812 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fae40f-607a-4dc2-ae0d-dd95b0256d6b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:52.246841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb8b7b-78a9-4601-93ec-6a0f8cb41033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:52.247535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb8b7b-78a9-4601-93ec-6a0f8cb41033 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:52.247938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb8b7b-78a9-4601-93ec-6a0f8cb41033 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:52.248219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb8b7b-78a9-4601-93ec-6a0f8cb41033 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:52.249063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb8b7b-78a9-4601-93ec-6a0f8cb41033 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:52.249373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb8b7b-78a9-4601-93ec-6a0f8cb41033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:52.250837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb8b7b-78a9-4601-93ec-6a0f8cb41033 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:52.258481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 359/705] 10.222.0.43 () {56 vars in 1124 bytes} [Mon Jul 4 07:50:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:52.259922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd9dd7-c236-46eb-8e56-c4900c8b5298 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:52.312860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55113ca-5edf-4a81-9139-2460fa5debb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:52.313549 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55113ca-5edf-4a81-9139-2460fa5debb5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:52.313865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55113ca-5edf-4a81-9139-2460fa5debb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:52.314202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55113ca-5edf-4a81-9139-2460fa5debb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:52.314949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55113ca-5edf-4a81-9139-2460fa5debb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:52.315258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55113ca-5edf-4a81-9139-2460fa5debb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:52.316681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55113ca-5edf-4a81-9139-2460fa5debb5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:52.331556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 347/706] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:52.337379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb8b7b-78a9-4601-93ec-6a0f8cb41033 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:52.426649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4105a9b-b1db-41c7-9020-9321562b28f8 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:52.426881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4105a9b-b1db-41c7-9020-9321562b28f8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:52.427106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4105a9b-b1db-41c7-9020-9321562b28f8 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:52.429029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 348/707] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:50:52.435827 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5667eb3-4b47-4aec-8024-c33e3a340c57 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:52.436126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5667eb3-4b47-4aec-8024-c33e3a340c57 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:52.436389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5667eb3-4b47-4aec-8024-c33e3a340c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:52.465583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f5667eb3-4b47-4aec-8024-c33e3a340c57 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:52.467242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f5667eb3-4b47-4aec-8024-c33e3a340c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:52.477363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb8b7b-78a9-4601-93ec-6a0f8cb41033 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:52.521971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d89c8-baa1-48d2-99c5-1154894eef9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:52.523674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d89c8-baa1-48d2-99c5-1154894eef9c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:52.524188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d89c8-baa1-48d2-99c5-1154894eef9c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:52.524669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d89c8-baa1-48d2-99c5-1154894eef9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:52.535705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d89c8-baa1-48d2-99c5-1154894eef9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:52.538286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d89c8-baa1-48d2-99c5-1154894eef9c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:52.549879 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:52.557109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 349/708] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-953d89c8-baa1-48d2-99c5-1154894eef9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:52.569689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:52.570671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:52.571703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:52.571703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:52.571703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:52.571703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:52.571703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:52.571703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:52.571703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:52.571703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:52.574411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 360/709] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:52.580807 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5667eb3-4b47-4aec-8024-c33e3a340c57 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:52.599311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d89c8-baa1-48d2-99c5-1154894eef9c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:52.630373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:52.744075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d89c8-baa1-48d2-99c5-1154894eef9c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:52.745263 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5667eb3-4b47-4aec-8024-c33e3a340c57 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:52.782844 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d26d6-332d-4512-b07b-0a1becf2ad37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:52.783542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d26d6-332d-4512-b07b-0a1becf2ad37 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:52.783822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d26d6-332d-4512-b07b-0a1becf2ad37 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:52.784097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d26d6-332d-4512-b07b-0a1becf2ad37 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:52.784888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d26d6-332d-4512-b07b-0a1becf2ad37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:52.785194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d26d6-332d-4512-b07b-0a1becf2ad37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:52.786699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d26d6-332d-4512-b07b-0a1becf2ad37 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:52.791464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b37117-07d0-4f05-9d49-5b025c8c2c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:52.792275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b37117-07d0-4f05-9d49-5b025c8c2c05 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:52.792556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b37117-07d0-4f05-9d49-5b025c8c2c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:52.792822 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b37117-07d0-4f05-9d49-5b025c8c2c05 None admin] PATH_INFO: `/v3/services/network` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:52.793475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b37117-07d0-4f05-9d49-5b025c8c2c05 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:52.795537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b37117-07d0-4f05-9d49-5b025c8c2c05 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:52.800563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 350/710] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:52.801949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9db90-6849-427d-9b75-7513a730e226 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:52.802235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9db90-6849-427d-9b75-7513a730e226 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:52.802509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9db90-6849-427d-9b75-7513a730e226 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:52.804051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 351/711] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:52.810551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9db90-6849-427d-9b75-7513a730e226 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c2b37117-07d0-4f05-9d49-5b025c8c2c05 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:52.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:52.815766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:52.816664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 361/712] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:52 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:52.817483 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e901c64-6524-4eed-a7e6-15db633fada1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:52.820007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e901c64-6524-4eed-a7e6-15db633fada1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:52.820726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e901c64-6524-4eed-a7e6-15db633fada1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:52.848639 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2e901c64-6524-4eed-a7e6-15db633fada1 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:52.850519 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2e901c64-6524-4eed-a7e6-15db633fada1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:52.898136 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:52.934275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 362/713] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:52.935161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e901c64-6524-4eed-a7e6-15db633fada1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:52.952953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9db90-6849-427d-9b75-7513a730e226 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:52.954385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:52.998464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46f519-4d76-48d2-a55c-43f323f64a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:53.000388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46f519-4d76-48d2-a55c-43f323f64a22 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:53.000875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46f519-4d76-48d2-a55c-43f323f64a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:53.001503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46f519-4d76-48d2-a55c-43f323f64a22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:53.002902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46f519-4d76-48d2-a55c-43f323f64a22 None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=15211e42ec9d4c1699d877f0b55ce918) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:53.003671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46f519-4d76-48d2-a55c-43f323f64a22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:53.006007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46f519-4d76-48d2-a55c-43f323f64a22 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:53.016226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 352/714] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:52 2022] GET /identity/v3/role_assignments?user.id=15211e42ec9d4c1699d877f0b55ce918&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:53.022001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46f519-4d76-48d2-a55c-43f323f64a22 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:53.086595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e901c64-6524-4eed-a7e6-15db633fada1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:53.182288 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc12ae6-14fe-414f-823d-ecc7adcab33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:53.182989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc12ae6-14fe-414f-823d-ecc7adcab33a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:53.183270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc12ae6-14fe-414f-823d-ecc7adcab33a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:53.183546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc12ae6-14fe-414f-823d-ecc7adcab33a None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:53.184276 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc12ae6-14fe-414f-823d-ecc7adcab33a None admin] RBAC: Adding query filter params (name=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:53.184584 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc12ae6-14fe-414f-823d-ecc7adcab33a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:53.186076 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc12ae6-14fe-414f-823d-ecc7adcab33a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:53.198270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 363/715] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:50:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:53.204766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc12ae6-14fe-414f-823d-ecc7adcab33a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:53.228026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46f519-4d76-48d2-a55c-43f323f64a22 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:53.257836 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:53.279956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438df1e-e15e-43f4-af4e-34266a65e999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:53.280823 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438df1e-e15e-43f4-af4e-34266a65e999 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:53.281107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438df1e-e15e-43f4-af4e-34266a65e999 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:53.281382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438df1e-e15e-43f4-af4e-34266a65e999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:53.302704 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438df1e-e15e-43f4-af4e-34266a65e999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:53.304261 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438df1e-e15e-43f4-af4e-34266a65e999 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b438df1e-e15e-43f4-af4e-34266a65e999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:53.331307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:53.332284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:53.333196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:53.333196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:53.333196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:53.333196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:53.333196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 353/716] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:53.342632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438df1e-e15e-43f4-af4e-34266a65e999 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:53.370644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc12ae6-14fe-414f-823d-ecc7adcab33a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:53.398071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:53.443306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24380d-09c5-4bc3-88a8-bf4b0845b38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:53.443989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24380d-09c5-4bc3-88a8-bf4b0845b38f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:53.444547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24380d-09c5-4bc3-88a8-bf4b0845b38f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:53.444943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24380d-09c5-4bc3-88a8-bf4b0845b38f None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:53.445686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24380d-09c5-4bc3-88a8-bf4b0845b38f None admin] RBAC: Adding query filter params (type=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:53.446000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24380d-09c5-4bc3-88a8-bf4b0845b38f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:53.447408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24380d-09c5-4bc3-88a8-bf4b0845b38f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:53.459983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 364/717] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:50:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:53.491273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438df1e-e15e-43f4-af4e-34266a65e999 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:53.537489 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc585b8-4d58-4140-a574-d044bdbf59f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:53.538363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc585b8-4d58-4140-a574-d044bdbf59f6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:53.538768 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc585b8-4d58-4140-a574-d044bdbf59f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:53.539136 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc585b8-4d58-4140-a574-d044bdbf59f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:53.539918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc585b8-4d58-4140-a574-d044bdbf59f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:53.540227 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc585b8-4d58-4140-a574-d044bdbf59f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:53.541739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc585b8-4d58-4140-a574-d044bdbf59f6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:53.568811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 354/718] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:53.575336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24380d-09c5-4bc3-88a8-bf4b0845b38f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:53.737644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24380d-09c5-4bc3-88a8-bf4b0845b38f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:53.784104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81598e3f-94ab-4af0-bad3-13498bfe2527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:53.784789 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81598e3f-94ab-4af0-bad3-13498bfe2527 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:53.785073 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81598e3f-94ab-4af0-bad3-13498bfe2527 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:53.785413 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81598e3f-94ab-4af0-bad3-13498bfe2527 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:53.786219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81598e3f-94ab-4af0-bad3-13498bfe2527 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:53.787732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81598e3f-94ab-4af0-bad3-13498bfe2527 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:53.817986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:53.829363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 365/719] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:50:53 2022] POST /identity/v3/users => generated 254 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:50:54.590576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332901e-5ce7-48ec-a8ba-b641d58045ac None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:54.591109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332901e-5ce7-48ec-a8ba-b641d58045ac None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:54.591109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332901e-5ce7-48ec-a8ba-b641d58045ac None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:54.592747 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 355/720] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:54.600975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016c69e-aa09-4003-ab0d-cd8e033c432f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:54.601647 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016c69e-aa09-4003-ab0d-cd8e033c432f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:54.601989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016c69e-aa09-4003-ab0d-cd8e033c432f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:54.634710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1016c69e-aa09-4003-ab0d-cd8e033c432f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:54.635923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1016c69e-aa09-4003-ab0d-cd8e033c432f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:54.667252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d57f40-8c44-44a1-95dd-b59a2e042c3c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:54.667874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d57f40-8c44-44a1-95dd-b59a2e042c3c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:54.668365 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d57f40-8c44-44a1-95dd-b59a2e042c3c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:54.671429 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 356/721] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:54.676225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01ad2d-b903-452e-aa31-aaa67df38f55 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:54.676850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01ad2d-b903-452e-aa31-aaa67df38f55 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:54.677324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01ad2d-b903-452e-aa31-aaa67df38f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:54.707358 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5a01ad2d-b903-452e-aa31-aaa67df38f55 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:54.708931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5a01ad2d-b903-452e-aa31-aaa67df38f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:54.757254 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 366/722] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:54.766027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:54.801749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016c69e-aa09-4003-ab0d-cd8e033c432f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:54.813215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 357/723] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:50:54.856820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01ad2d-b903-452e-aa31-aaa67df38f55 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:54.995765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016c69e-aa09-4003-ab0d-cd8e033c432f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:55.055104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01ad2d-b903-452e-aa31-aaa67df38f55 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:55.055787 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:55.074398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f557a76-3aa5-4d64-95c6-6a3d3d7b91bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:55.075090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f557a76-3aa5-4d64-95c6-6a3d3d7b91bb None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:55.075369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f557a76-3aa5-4d64-95c6-6a3d3d7b91bb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:55.075643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f557a76-3aa5-4d64-95c6-6a3d3d7b91bb None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:55.076378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f557a76-3aa5-4d64-95c6-6a3d3d7b91bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:55.077806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f557a76-3aa5-4d64-95c6-6a3d3d7b91bb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:55.107714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 367/724] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:50:54 2022] POST /identity/v3/services => generated 251 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 07:50:55.139169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e1a67-1395-470d-81c0-79a30bc581cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:55.140011 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e1a67-1395-470d-81c0-79a30bc581cd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:55.140292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e1a67-1395-470d-81c0-79a30bc581cd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:55.140568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e1a67-1395-470d-81c0-79a30bc581cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:55.163838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e1a67-1395-470d-81c0-79a30bc581cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-016e1a67-1395-470d-81c0-79a30bc581cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:55.175971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:55.177016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:55.177949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 358/725] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:50:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:55.184376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f557a76-3aa5-4d64-95c6-6a3d3d7b91bb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:55.312667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f557a76-3aa5-4d64-95c6-6a3d3d7b91bb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:55.365914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc2bd3-71f1-4883-b346-a65db55e34be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:55.366617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc2bd3-71f1-4883-b346-a65db55e34be None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:55.366898 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc2bd3-71f1-4883-b346-a65db55e34be None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:55.367245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc2bd3-71f1-4883-b346-a65db55e34be None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:55.367926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc2bd3-71f1-4883-b346-a65db55e34be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:55.368230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc2bd3-71f1-4883-b346-a65db55e34be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:55.369650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc2bd3-71f1-4883-b346-a65db55e34be None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:55.379869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 368/726] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:55.386196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e1a67-1395-470d-81c0-79a30bc581cd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:55.429992 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:55.505421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e1a67-1395-470d-81c0-79a30bc581cd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:55.527931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906d960-4e46-4563-9b8d-d711c2f97a9e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:55.528257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906d960-4e46-4563-9b8d-d711c2f97a9e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:55.528515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906d960-4e46-4563-9b8d-d711c2f97a9e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:55.530234 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 369/727] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:50:55.535552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d73d96-6881-4fb2-b9d8-f5f2f73a0405 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:55.535834 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d73d96-6881-4fb2-b9d8-f5f2f73a0405 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:55.536113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d73d96-6881-4fb2-b9d8-f5f2f73a0405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:55.555386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50111a-c9ac-4513-9535-6828f8394177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:55.556175 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-04d73d96-6881-4fb2-b9d8-f5f2f73a0405 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:55.556310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50111a-c9ac-4513-9535-6828f8394177 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:55.556478 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50111a-c9ac-4513-9535-6828f8394177 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:55.556732 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50111a-c9ac-4513-9535-6828f8394177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:55.558107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-04d73d96-6881-4fb2-b9d8-f5f2f73a0405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:55.569417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50111a-c9ac-4513-9535-6828f8394177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:55.571070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50111a-c9ac-4513-9535-6828f8394177 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-fa50111a-c9ac-4513-9535-6828f8394177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:55.590916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:55.592323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:55.593240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:55.593240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:55.593240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:55.593240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:55.593240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 359/728] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:55.598933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50111a-c9ac-4513-9535-6828f8394177 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:55.622004 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:55.647088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 370/729] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 07:50:55.674056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d73d96-6881-4fb2-b9d8-f5f2f73a0405 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:55.705708 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:55.753205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50111a-c9ac-4513-9535-6828f8394177 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:55.771602 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d73d96-6881-4fb2-b9d8-f5f2f73a0405 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:55.813163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b68f00-2c55-4991-b536-2f505e10c2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:55.814039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b68f00-2c55-4991-b536-2f505e10c2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:55.814317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b68f00-2c55-4991-b536-2f505e10c2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:55.814587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b68f00-2c55-4991-b536-2f505e10c2d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:55.827815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b68f00-2c55-4991-b536-2f505e10c2d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b0b68f00-2c55-4991-b536-2f505e10c2d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:55.836253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:55.837362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:55.838347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 371/730] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:50:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:55.850771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b68f00-2c55-4991-b536-2f505e10c2d1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:55.862266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9de42-0a32-4059-b910-eb13bf051e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:55.863400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9de42-0a32-4059-b910-eb13bf051e98 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:55.863890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9de42-0a32-4059-b910-eb13bf051e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:55.864454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9de42-0a32-4059-b910-eb13bf051e98 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:55.865455 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9de42-0a32-4059-b910-eb13bf051e98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:55.866126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9de42-0a32-4059-b910-eb13bf051e98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:55.867921 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9de42-0a32-4059-b910-eb13bf051e98 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:55.891097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 360/731] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:55.897528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:55.898091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9de42-0a32-4059-b910-eb13bf051e98 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:56.012701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b68f00-2c55-4991-b536-2f505e10c2d1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:56.038830 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9de42-0a32-4059-b910-eb13bf051e98 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:56.063422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aab09c-cdc9-4a90-b763-c045df44a18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:56.064155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aab09c-cdc9-4a90-b763-c045df44a18e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:56.064434 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aab09c-cdc9-4a90-b763-c045df44a18e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:56.064740 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aab09c-cdc9-4a90-b763-c045df44a18e None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:56.065778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aab09c-cdc9-4a90-b763-c045df44a18e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:56.065898 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aab09c-cdc9-4a90-b763-c045df44a18e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:56.067314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aab09c-cdc9-4a90-b763-c045df44a18e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:56.076797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd02f3e-280e-42bd-ac70-89e38efb2e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:56.081238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 372/732] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:50:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:56.081238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d376-b236-4106-9738-f646b62de7d5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:56.081238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d376-b236-4106-9738-f646b62de7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:56.081238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d376-b236-4106-9738-f646b62de7d5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:56.081238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 373/733] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:56.082194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd02f3e-280e-42bd-ac70-89e38efb2e88 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:56.082709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd02f3e-280e-42bd-ac70-89e38efb2e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:56.083183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd02f3e-280e-42bd-ac70-89e38efb2e88 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:56.087020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:56.091784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64484217-24cb-4923-acc6-affc5a799ad8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:56.092074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64484217-24cb-4923-acc6-affc5a799ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:56.092351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64484217-24cb-4923-acc6-affc5a799ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:56.096645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd02f3e-280e-42bd-ac70-89e38efb2e88 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:56.098458 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd02f3e-280e-42bd-ac70-89e38efb2e88 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:56.110681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-64484217-24cb-4923-acc6-affc5a799ad8 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:56.112104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-64484217-24cb-4923-acc6-affc5a799ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0dd02f3e-280e-42bd-ac70-89e38efb2e88 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:56.118874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:56.120002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:56.121214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:56.121214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:56.121214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:56.121214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:56.121214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:56.121214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:56.121214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:56.121214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:56.121214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 07:50:56.121214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:56.121214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 361/734] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:50:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:56.126061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd02f3e-280e-42bd-ac70-89e38efb2e88 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:56.216284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 374/735] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 07:50:56.217423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64484217-24cb-4923-acc6-affc5a799ad8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:56.253066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:56.260744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd02f3e-280e-42bd-ac70-89e38efb2e88 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:56.298340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c649db-bcb5-40f1-9ef3-76f2d8c7db0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:56.299551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c649db-bcb5-40f1-9ef3-76f2d8c7db0c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:56.300064 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c649db-bcb5-40f1-9ef3-76f2d8c7db0c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:56.300536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c649db-bcb5-40f1-9ef3-76f2d8c7db0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:56.312040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c649db-bcb5-40f1-9ef3-76f2d8c7db0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:56.313901 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c649db-bcb5-40f1-9ef3-76f2d8c7db0c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:56.327627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64484217-24cb-4923-acc6-affc5a799ad8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-92c649db-bcb5-40f1-9ef3-76f2d8c7db0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:56.332911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:56.333862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:56.334888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:56.334888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:56.334888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:56.334888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:56.341749 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 362/736] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:56.342937 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c649db-bcb5-40f1-9ef3-76f2d8c7db0c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:56.365098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746300d-25cc-4d24-a91f-1e8f5f5b5287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:56.365814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746300d-25cc-4d24-a91f-1e8f5f5b5287 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:56.366252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746300d-25cc-4d24-a91f-1e8f5f5b5287 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:56.366692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746300d-25cc-4d24-a91f-1e8f5f5b5287 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:56.367582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746300d-25cc-4d24-a91f-1e8f5f5b5287 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:56.367889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746300d-25cc-4d24-a91f-1e8f5f5b5287 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:56.369424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746300d-25cc-4d24-a91f-1e8f5f5b5287 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:56.389208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 375/737] 10.222.0.43 () {56 vars in 1135 bytes} [Mon Jul 4 07:50:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:56.394573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746300d-25cc-4d24-a91f-1e8f5f5b5287 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:56.396883 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:56.455861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c649db-bcb5-40f1-9ef3-76f2d8c7db0c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:56.494309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:56.495642 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f941487-7c6f-4b63-ab01-55af603d0c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:56.497097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f941487-7c6f-4b63-ab01-55af603d0c17 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:56.497959 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f941487-7c6f-4b63-ab01-55af603d0c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:56.498719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f941487-7c6f-4b63-ab01-55af603d0c17 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:56.500024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f941487-7c6f-4b63-ab01-55af603d0c17 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:56.501775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f941487-7c6f-4b63-ab01-55af603d0c17 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:56.506334 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746300d-25cc-4d24-a91f-1e8f5f5b5287 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:56.524228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 363/738] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:50:56 2022] POST /identity/v3/services => generated 240 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:50:56.526998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f941487-7c6f-4b63-ab01-55af603d0c17 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:56.581705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d1cac-e4fe-4d66-99ec-2c132aa06efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:56.582841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d1cac-e4fe-4d66-99ec-2c132aa06efc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:56.583388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d1cac-e4fe-4d66-99ec-2c132aa06efc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:56.583874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d1cac-e4fe-4d66-99ec-2c132aa06efc None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:56.584861 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d1cac-e4fe-4d66-99ec-2c132aa06efc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:56.585402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d1cac-e4fe-4d66-99ec-2c132aa06efc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:56.588613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d1cac-e4fe-4d66-99ec-2c132aa06efc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:56.617907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 376/739] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:56.631889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d1cac-e4fe-4d66-99ec-2c132aa06efc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:56.742305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f941487-7c6f-4b63-ab01-55af603d0c17 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:56.754803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:56.788692 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c7b9e-b122-48f6-a51b-cabcfc2741f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:56.789383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c7b9e-b122-48f6-a51b-cabcfc2741f8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:56.790080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c7b9e-b122-48f6-a51b-cabcfc2741f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:56.790080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c7b9e-b122-48f6-a51b-cabcfc2741f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:56.826982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d1cac-e4fe-4d66-99ec-2c132aa06efc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:56.863684 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29610b-e85f-4268-b38b-78723444a293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:56.864497 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29610b-e85f-4268-b38b-78723444a293 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:56.864780 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29610b-e85f-4268-b38b-78723444a293 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:56.865060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29610b-e85f-4268-b38b-78723444a293 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:56.873739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29610b-e85f-4268-b38b-78723444a293 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:56.875212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29610b-e85f-4268-b38b-78723444a293 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:56.879657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c7b9e-b122-48f6-a51b-cabcfc2741f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:56.881379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c7b9e-b122-48f6-a51b-cabcfc2741f8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6a29610b-e85f-4268-b38b-78723444a293 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:56.890515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:56.891530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:56.893106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:56.893106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:56.893106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:56.893106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:56.893106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:56.893106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:56.893106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:56.893106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:56.893106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:50:56.893106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:56.893106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 377/740] 10.222.0.43 () {56 vars in 1093 bytes} [Mon Jul 4 07:50:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:50:56.893776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87456ec8-3b81-4d89-b9b6-617e6fd0a75d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:56.893776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87456ec8-3b81-4d89-b9b6-617e6fd0a75d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:56.893917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87456ec8-3b81-4d89-b9b6-617e6fd0a75d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:56.895391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 378/741] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:56.900328 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87456ec8-3b81-4d89-b9b6-617e6fd0a75d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:56.913786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:56.998569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 364/742] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:50:57.000442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a29b3-fc94-4e0e-8b6f-8a99497d535a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:57.000978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a29b3-fc94-4e0e-8b6f-8a99497d535a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:57.001518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a29b3-fc94-4e0e-8b6f-8a99497d535a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:57.014719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:57.017297 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87456ec8-3b81-4d89-b9b6-617e6fd0a75d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:57.022110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-039a29b3-fc94-4e0e-8b6f-8a99497d535a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:57.024127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-039a29b3-fc94-4e0e-8b6f-8a99497d535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:57.056347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc290d24-139c-489f-983e-5fad60c25573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:57.057327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc290d24-139c-489f-983e-5fad60c25573 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:57.057927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc290d24-139c-489f-983e-5fad60c25573 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:57.058376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc290d24-139c-489f-983e-5fad60c25573 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:57.059514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc290d24-139c-489f-983e-5fad60c25573 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:57.059977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc290d24-139c-489f-983e-5fad60c25573 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:57.061949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc290d24-139c-489f-983e-5fad60c25573 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:57.083003 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 379/743] 10.222.0.43 () {56 vars in 1133 bytes} [Mon Jul 4 07:50:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:57.084088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc290d24-139c-489f-983e-5fad60c25573 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:57.109770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 365/744] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:50:57.111885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a29b3-fc94-4e0e-8b6f-8a99497d535a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:57.165099 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:57.211767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc290d24-139c-489f-983e-5fad60c25573 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:57.251619 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b757cf-6a2a-456c-8fea-3a1ec7f0573b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:57.252577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b757cf-6a2a-456c-8fea-3a1ec7f0573b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:57.252881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b757cf-6a2a-456c-8fea-3a1ec7f0573b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:57.253139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b757cf-6a2a-456c-8fea-3a1ec7f0573b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:57.267859 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b757cf-6a2a-456c-8fea-3a1ec7f0573b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:57.269744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b757cf-6a2a-456c-8fea-3a1ec7f0573b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-71b757cf-6a2a-456c-8fea-3a1ec7f0573b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:57.290407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:57.291399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:57.292411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:57.292411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:57.292411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:57.292411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:57.294827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 380/745] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:57.296037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b757cf-6a2a-456c-8fea-3a1ec7f0573b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:57.297137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a29b3-fc94-4e0e-8b6f-8a99497d535a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:57.338542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48d37f-aab1-485d-ac8b-6b8021f61fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:57.340747 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48d37f-aab1-485d-ac8b-6b8021f61fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:57.341222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48d37f-aab1-485d-ac8b-6b8021f61fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:57.341709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48d37f-aab1-485d-ac8b-6b8021f61fa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:57.345873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:57.358053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48d37f-aab1-485d-ac8b-6b8021f61fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:57.360304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48d37f-aab1-485d-ac8b-6b8021f61fa9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0c48d37f-aab1-485d-ac8b-6b8021f61fa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:57.385913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:57.386886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:57.387793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:57.387793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:57.387793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:57.387793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:57.390498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 366/746] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:57.392068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48d37f-aab1-485d-ac8b-6b8021f61fa9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:57.429426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b757cf-6a2a-456c-8fea-3a1ec7f0573b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:57.459905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:57.486016 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dc795-70a9-4cc0-a862-0051652b9941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:57.487424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dc795-70a9-4cc0-a862-0051652b9941 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:57.487874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dc795-70a9-4cc0-a862-0051652b9941 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:57.488290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dc795-70a9-4cc0-a862-0051652b9941 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:57.504567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799dc795-70a9-4cc0-a862-0051652b9941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:50:57.508328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48d37f-aab1-485d-ac8b-6b8021f61fa9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-799dc795-70a9-4cc0-a862-0051652b9941 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:50:57.515337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:57.516646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:57.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:57.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:57.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:57.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:57.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:50:57.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:50:57.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:50:57.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:50:57.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:50:57.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:50:57.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:50:57.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:57.525813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 381/747] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:50:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:57.525813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dc795-70a9-4cc0-a862-0051652b9941 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:57.566683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34df8a-9bf5-460e-8ad8-0111f326a2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:57.567771 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34df8a-9bf5-460e-8ad8-0111f326a2ea None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:57.568550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34df8a-9bf5-460e-8ad8-0111f326a2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:57.569114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34df8a-9bf5-460e-8ad8-0111f326a2ea None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:57.570688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34df8a-9bf5-460e-8ad8-0111f326a2ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:57.571020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34df8a-9bf5-460e-8ad8-0111f326a2ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:57.572535 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34df8a-9bf5-460e-8ad8-0111f326a2ea None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:57.586574 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 367/748] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:57.587483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34df8a-9bf5-460e-8ad8-0111f326a2ea None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:57.644549 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:57.673882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dc795-70a9-4cc0-a862-0051652b9941 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:57.709415 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34df8a-9bf5-460e-8ad8-0111f326a2ea None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:57.725455 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f86f2c-dbba-4ef8-a2f8-30da16799bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:57.726196 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f86f2c-dbba-4ef8-a2f8-30da16799bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:57.726540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f86f2c-dbba-4ef8-a2f8-30da16799bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:57.726747 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f86f2c-dbba-4ef8-a2f8-30da16799bc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:57.727480 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f86f2c-dbba-4ef8-a2f8-30da16799bc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:57.727781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f86f2c-dbba-4ef8-a2f8-30da16799bc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:57.729172 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f86f2c-dbba-4ef8-a2f8-30da16799bc8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:57.742500 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 382/749] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:57.743438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f86f2c-dbba-4ef8-a2f8-30da16799bc8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:57.757836 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0db7ea-06fb-46cd-a872-b1db016113f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:57.758532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0db7ea-06fb-46cd-a872-b1db016113f5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:57.758795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0db7ea-06fb-46cd-a872-b1db016113f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:57.759074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0db7ea-06fb-46cd-a872-b1db016113f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:57.759824 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0db7ea-06fb-46cd-a872-b1db016113f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:57.760130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0db7ea-06fb-46cd-a872-b1db016113f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:57.761536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0db7ea-06fb-46cd-a872-b1db016113f5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:57.769902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 368/750] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:50:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:57.770822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0db7ea-06fb-46cd-a872-b1db016113f5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:57.795591 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:57.860078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f86f2c-dbba-4ef8-a2f8-30da16799bc8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:57.885551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0db7ea-06fb-46cd-a872-b1db016113f5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:57.897086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:57.897731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca804f-496b-4b42-825f-fc1887d079cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:57.898650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca804f-496b-4b42-825f-fc1887d079cf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:57.898818 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca804f-496b-4b42-825f-fc1887d079cf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:57.899196 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca804f-496b-4b42-825f-fc1887d079cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:57.907226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca804f-496b-4b42-825f-fc1887d079cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:57.908817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca804f-496b-4b42-825f-fc1887d079cf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:57.923324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c2522-6dd5-40b5-88ce-6e6cf925aa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9fca804f-496b-4b42-825f-fc1887d079cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:57.923919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:57.924869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:57.926253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:57.926253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:57.926253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:57.926253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:57.926253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:57.926253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:57.926253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:57.926253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:57.926729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c2522-6dd5-40b5-88ce-6e6cf925aa70 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:57.926729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c2522-6dd5-40b5-88ce-6e6cf925aa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:57.926729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c2522-6dd5-40b5-88ce-6e6cf925aa70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:57.926940 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 383/751] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:57.926940 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254d999-add4-43ae-8228-131c236fac04 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:57.927073 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254d999-add4-43ae-8228-131c236fac04 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:57.927399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254d999-add4-43ae-8228-131c236fac04 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:57.928753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 384/752] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:57.929367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254d999-add4-43ae-8228-131c236fac04 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:57.942259 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c2522-6dd5-40b5-88ce-6e6cf925aa70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:57.944418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c2522-6dd5-40b5-88ce-6e6cf925aa70 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-639c2522-6dd5-40b5-88ce-6e6cf925aa70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:50:57.964765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:57.965904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:57.966820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:50:57.966820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:50:57.966820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:50:57.966820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:50:57.966820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:50:57.966820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:50:57.966820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:50:57.966820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:57.967948 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 369/753] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:57.968798 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c2522-6dd5-40b5-88ce-6e6cf925aa70 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:58.051203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:58.057580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254d999-add4-43ae-8228-131c236fac04 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:58.074138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c2522-6dd5-40b5-88ce-6e6cf925aa70 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:58.099222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8def57-8be9-4004-9352-84ff6b9206f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:58.100581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8def57-8be9-4004-9352-84ff6b9206f0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:58.101444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8def57-8be9-4004-9352-84ff6b9206f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:58.102067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8def57-8be9-4004-9352-84ff6b9206f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:58.109361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c05f78-988c-4df0-b5f7-9576dbf4cd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:58.110087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c05f78-988c-4df0-b5f7-9576dbf4cd4c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:58.110390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c05f78-988c-4df0-b5f7-9576dbf4cd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:58.110656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c05f78-988c-4df0-b5f7-9576dbf4cd4c None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:58.111484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c05f78-988c-4df0-b5f7-9576dbf4cd4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:58.111799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c05f78-988c-4df0-b5f7-9576dbf4cd4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:58.113269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c05f78-988c-4df0-b5f7-9576dbf4cd4c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:58.125059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8def57-8be9-4004-9352-84ff6b9206f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:58.126581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8def57-8be9-4004-9352-84ff6b9206f0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:58.132961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 370/754] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:58.132961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495f67d-a4fb-4f72-adce-0b9bda296324 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:58.132961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495f67d-a4fb-4f72-adce-0b9bda296324 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:58.132961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495f67d-a4fb-4f72-adce-0b9bda296324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:58.148700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9495f67d-a4fb-4f72-adce-0b9bda296324 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:58.149945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9495f67d-a4fb-4f72-adce-0b9bda296324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8b8def57-8be9-4004-9352-84ff6b9206f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:58.150654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:58.151577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:58.158152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:58.158152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:58.158152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:58.158152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:50:58.158152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 385/755] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:58.158152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8def57-8be9-4004-9352-84ff6b9206f0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:58.185310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:58.230608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 371/756] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:50:58.231803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495f67d-a4fb-4f72-adce-0b9bda296324 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:58.278770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8def57-8be9-4004-9352-84ff6b9206f0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:58.315531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896bdc1-76ef-4e8a-b1ce-fe7d46f81e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:58.316217 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896bdc1-76ef-4e8a-b1ce-fe7d46f81e47 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:58.316557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896bdc1-76ef-4e8a-b1ce-fe7d46f81e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:58.316898 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896bdc1-76ef-4e8a-b1ce-fe7d46f81e47 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:58.317786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896bdc1-76ef-4e8a-b1ce-fe7d46f81e47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:58.318152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896bdc1-76ef-4e8a-b1ce-fe7d46f81e47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:58.319650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896bdc1-76ef-4e8a-b1ce-fe7d46f81e47 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:58.332211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 386/757] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:50:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:58.333036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896bdc1-76ef-4e8a-b1ce-fe7d46f81e47 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:58.342934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495f67d-a4fb-4f72-adce-0b9bda296324 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:58.347769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:58.381345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a3da1-f3f1-446f-8349-63a373521c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:58.382400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a3da1-f3f1-446f-8349-63a373521c4b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:58.382828 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a3da1-f3f1-446f-8349-63a373521c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:58.383108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a3da1-f3f1-446f-8349-63a373521c4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:58.383943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a3da1-f3f1-446f-8349-63a373521c4b None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=6763884b51204c488ca8c7e452e3a29e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:58.384254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a3da1-f3f1-446f-8349-63a373521c4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:58.385727 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a3da1-f3f1-446f-8349-63a373521c4b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:58.401858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 372/758] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:50:58 2022] GET /identity/v3/role_assignments?user.id=6763884b51204c488ca8c7e452e3a29e&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:58.403572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a3da1-f3f1-446f-8349-63a373521c4b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:58.441455 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:58.457171 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896bdc1-76ef-4e8a-b1ce-fe7d46f81e47 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:58.510390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d80e2-b88e-49c1-aebf-cda4a53a5522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:58.511454 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d80e2-b88e-49c1-aebf-cda4a53a5522 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:58.511937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d80e2-b88e-49c1-aebf-cda4a53a5522 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:58.512389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d80e2-b88e-49c1-aebf-cda4a53a5522 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:58.513322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d80e2-b88e-49c1-aebf-cda4a53a5522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:58.513844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d80e2-b88e-49c1-aebf-cda4a53a5522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:58.516403 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d80e2-b88e-49c1-aebf-cda4a53a5522 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:58.535105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 387/759] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:58.536320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d80e2-b88e-49c1-aebf-cda4a53a5522 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:58.545207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a3da1-f3f1-446f-8349-63a373521c4b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:58.586163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671319c2-71f1-41f3-bbc6-a3ceb1c72bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:58.587584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671319c2-71f1-41f3-bbc6-a3ceb1c72bce None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:58.587962 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671319c2-71f1-41f3-bbc6-a3ceb1c72bce None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:58.588235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671319c2-71f1-41f3-bbc6-a3ceb1c72bce None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:58.588903 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671319c2-71f1-41f3-bbc6-a3ceb1c72bce None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:58.590738 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671319c2-71f1-41f3-bbc6-a3ceb1c72bce None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-671319c2-71f1-41f3-bbc6-a3ceb1c72bce None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:58.609785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:58.611214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 07:50:58.612101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:58.617220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 373/760] 10.222.0.43 () {56 vars in 1129 bytes} [Mon Jul 4 07:50:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:58.618404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671319c2-71f1-41f3-bbc6-a3ceb1c72bce None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:58.647719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:58.687145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d80e2-b88e-49c1-aebf-cda4a53a5522 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:58.727124 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:58.735365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21e632-3080-4935-aebe-4eaa18b58684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:58.736532 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21e632-3080-4935-aebe-4eaa18b58684 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:58.736812 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21e632-3080-4935-aebe-4eaa18b58684 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:58.737088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21e632-3080-4935-aebe-4eaa18b58684 None admin] PATH_INFO: `/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/15211e42ec9d4c1699d877f0b55ce918/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:58.740049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21e632-3080-4935-aebe-4eaa18b58684 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f2d85c9e3d14bdabfb63fe505d83714, user_id=15211e42ec9d4c1699d877f0b55ce918, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:58.741997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21e632-3080-4935-aebe-4eaa18b58684 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:58.744572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671319c2-71f1-41f3-bbc6-a3ceb1c72bce None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:58.763191 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 388/761] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:50:58 2022] PUT /identity/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/15211e42ec9d4c1699d877f0b55ce918/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:50:58.764059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21e632-3080-4935-aebe-4eaa18b58684 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:58.787095 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af81dd-c295-4a9d-aa52-0c078fb1cf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:58.788796 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af81dd-c295-4a9d-aa52-0c078fb1cf24 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:58.789362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af81dd-c295-4a9d-aa52-0c078fb1cf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:58.789864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af81dd-c295-4a9d-aa52-0c078fb1cf24 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:58.799871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af81dd-c295-4a9d-aa52-0c078fb1cf24 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:58.801975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af81dd-c295-4a9d-aa52-0c078fb1cf24 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-52af81dd-c295-4a9d-aa52-0c078fb1cf24 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:50:58.819954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:58.821031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:58.822090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:58.822090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:58.822090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:50:58.822090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:50:58.822090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:50:58.822090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:50:58.822090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:50:58.822090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:50:58.822090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 07:50:58.822090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:58.824866 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 374/762] 10.222.0.43 () {56 vars in 1117 bytes} [Mon Jul 4 07:50:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:58.830817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af81dd-c295-4a9d-aa52-0c078fb1cf24 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:58.896494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21e632-3080-4935-aebe-4eaa18b58684 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:58.935569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:58.939205 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f2376-f97a-4dec-8b76-cbca066eeaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:58.939899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f2376-f97a-4dec-8b76-cbca066eeaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:58.940179 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f2376-f97a-4dec-8b76-cbca066eeaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:58.940453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f2376-f97a-4dec-8b76-cbca066eeaf0 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:58.941175 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f2376-f97a-4dec-8b76-cbca066eeaf0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:58.941479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f2376-f97a-4dec-8b76-cbca066eeaf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:58.942977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f2376-f97a-4dec-8b76-cbca066eeaf0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:58.948942 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 389/763] 10.222.0.43 () {56 vars in 1124 bytes} [Mon Jul 4 07:50:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:58.954597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977c0e-2359-4cc1-a187-93f65e7eb59f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:58.955468 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977c0e-2359-4cc1-a187-93f65e7eb59f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:58.955760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977c0e-2359-4cc1-a187-93f65e7eb59f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:58.957521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 390/764] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:50:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:50:58.958391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb977c0e-2359-4cc1-a187-93f65e7eb59f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:58.971475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af81dd-c295-4a9d-aa52-0c078fb1cf24 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:59.019549 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605ff84-3517-440b-b36b-3280c8924511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:59.020603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605ff84-3517-440b-b36b-3280c8924511 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:59.020892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605ff84-3517-440b-b36b-3280c8924511 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:59.021264 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605ff84-3517-440b-b36b-3280c8924511 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:59.022132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2605ff84-3517-440b-b36b-3280c8924511 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:59.022597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2605ff84-3517-440b-b36b-3280c8924511 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:59.024462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2605ff84-3517-440b-b36b-3280c8924511 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:59.043752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 375/765] 10.222.0.43 () {56 vars in 1149 bytes} [Mon Jul 4 07:50:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:59.045190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dd372-acfb-4fca-af74-68448eaa24fa None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:59.045480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dd372-acfb-4fca-af74-68448eaa24fa None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:59.049860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dd372-acfb-4fca-af74-68448eaa24fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:59.065311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:59.075810 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-333dd372-acfb-4fca-af74-68448eaa24fa None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:50:59.077001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-333dd372-acfb-4fca-af74-68448eaa24fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:50:59.090823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb977c0e-2359-4cc1-a187-93f65e7eb59f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:59.144205 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b548a-9d68-463a-98c6-4a7faf99562b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:59.144723 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b548a-9d68-463a-98c6-4a7faf99562b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:59.145103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b548a-9d68-463a-98c6-4a7faf99562b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:59.145535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b548a-9d68-463a-98c6-4a7faf99562b None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:59.146405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b548a-9d68-463a-98c6-4a7faf99562b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:59.146902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b548a-9d68-463a-98c6-4a7faf99562b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:59.148648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b548a-9d68-463a-98c6-4a7faf99562b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:59.154657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 391/766] 10.222.0.43 () {56 vars in 1124 bytes} [Mon Jul 4 07:50:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:59.160306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b548a-9d68-463a-98c6-4a7faf99562b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:59.169058 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 376/767] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:50:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:50:59.169058 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dd372-acfb-4fca-af74-68448eaa24fa None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:59.248285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:59.310224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b548a-9d68-463a-98c6-4a7faf99562b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:59.320717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dd372-acfb-4fca-af74-68448eaa24fa None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:59.347994 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc06ae-a5fd-4efc-b48e-5dedafddb3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:59.348660 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc06ae-a5fd-4efc-b48e-5dedafddb3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:59.348931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc06ae-a5fd-4efc-b48e-5dedafddb3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:59.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc06ae-a5fd-4efc-b48e-5dedafddb3f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:59.372605 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15537ad-30ce-4433-bb11-b0c66338b643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:59.373984 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15537ad-30ce-4433-bb11-b0c66338b643 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:59.374477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15537ad-30ce-4433-bb11-b0c66338b643 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:59.375104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15537ad-30ce-4433-bb11-b0c66338b643 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:59.376724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15537ad-30ce-4433-bb11-b0c66338b643 None admin] RBAC: Adding query filter params (interface=public, service_id=31ff52a0acec453c8d84034880617e12, region_id=RegionOne) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:50:59.377266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15537ad-30ce-4433-bb11-b0c66338b643 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:59.379896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15537ad-30ce-4433-bb11-b0c66338b643 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:59.386942 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:59.396504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 377/768] 10.222.0.43 () {56 vars in 1251 bytes} [Mon Jul 4 07:50:59 2022] GET /identity/v3/endpoints?service_id=31ff52a0acec453c8d84034880617e12&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:59.396504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15537ad-30ce-4433-bb11-b0c66338b643 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:59.428338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc06ae-a5fd-4efc-b48e-5dedafddb3f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:59.429811 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc06ae-a5fd-4efc-b48e-5dedafddb3f5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:59.494181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:59.523710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15537ad-30ce-4433-bb11-b0c66338b643 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:59.553779 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 392/769] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:50:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:50:59.554932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc06ae-a5fd-4efc-b48e-5dedafddb3f5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:59.562003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365272bd-98fc-4a35-88af-2dca48f609ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:59.563138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365272bd-98fc-4a35-88af-2dca48f609ac None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:59.563614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365272bd-98fc-4a35-88af-2dca48f609ac None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:59.564076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365272bd-98fc-4a35-88af-2dca48f609ac None admin] PATH_INFO: `/v3/services/network` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:59.565043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365272bd-98fc-4a35-88af-2dca48f609ac None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:59.566807 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365272bd-98fc-4a35-88af-2dca48f609ac None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-365272bd-98fc-4a35-88af-2dca48f609ac None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:59.580197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:59.581303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 07:50:59.582235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:59.586209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 378/770] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:50:59 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:59.587582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365272bd-98fc-4a35-88af-2dca48f609ac None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:59.664866 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:59.707626 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365272bd-98fc-4a35-88af-2dca48f609ac None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:59.713139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc06ae-a5fd-4efc-b48e-5dedafddb3f5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:50:59.747925 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be66b0e-10ac-479e-a88c-da635d6b1fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:59.748737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be66b0e-10ac-479e-a88c-da635d6b1fce None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:59.749018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be66b0e-10ac-479e-a88c-da635d6b1fce None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:59.749298 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be66b0e-10ac-479e-a88c-da635d6b1fce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:59.761409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be66b0e-10ac-479e-a88c-da635d6b1fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:59.762925 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be66b0e-10ac-479e-a88c-da635d6b1fce None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:59.766883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026865f3-81d2-48ea-8d21-02c76df4a98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:50:59.767821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026865f3-81d2-48ea-8d21-02c76df4a98e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:50:59.768313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026865f3-81d2-48ea-8d21-02c76df4a98e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:50:59.768791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026865f3-81d2-48ea-8d21-02c76df4a98e None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:50:59.770172 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026865f3-81d2-48ea-8d21-02c76df4a98e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:50:59.771581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026865f3-81d2-48ea-8d21-02c76df4a98e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:50:59.785666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 393/771] 10.222.0.43 () {56 vars in 1087 bytes} [Mon Jul 4 07:50:59 2022] GET /identity/v3/services => generated 772 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:50:59.785666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026865f3-81d2-48ea-8d21-02c76df4a98e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-6be66b0e-10ac-479e-a88c-da635d6b1fce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:50:59.791899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:50:59.793245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:50:59.794191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:50:59.794191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:50:59.794191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:50:59.794191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:50:59.799628 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 379/772] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:50:59.800055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be66b0e-10ac-479e-a88c-da635d6b1fce None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:50:59.834043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:50:59.993781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026865f3-81d2-48ea-8d21-02c76df4a98e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:00.011822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be66b0e-10ac-479e-a88c-da635d6b1fce None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:00.032075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:00.040183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e3d4d-a6c1-4940-9c53-c870d0cfb996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:00.041228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e3d4d-a6c1-4940-9c53-c870d0cfb996 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:00.041759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e3d4d-a6c1-4940-9c53-c870d0cfb996 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:00.042248 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e3d4d-a6c1-4940-9c53-c870d0cfb996 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:00.043215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e3d4d-a6c1-4940-9c53-c870d0cfb996 None admin] RBAC: Adding query filter params (name=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:00.043727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e3d4d-a6c1-4940-9c53-c870d0cfb996 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:00.045441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e3d4d-a6c1-4940-9c53-c870d0cfb996 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:00.051308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 394/773] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:50:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:00.057287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e64a4e-a3b2-4a75-beb8-25e793a12df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:00.059213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e3d4d-a6c1-4940-9c53-c870d0cfb996 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:00.060170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e64a4e-a3b2-4a75-beb8-25e793a12df0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:00.060523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e64a4e-a3b2-4a75-beb8-25e793a12df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:00.060889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e64a4e-a3b2-4a75-beb8-25e793a12df0 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:00.061657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e64a4e-a3b2-4a75-beb8-25e793a12df0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:00.062083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e64a4e-a3b2-4a75-beb8-25e793a12df0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:00.064025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e64a4e-a3b2-4a75-beb8-25e793a12df0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:00.080040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 380/774] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:00.085157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e64a4e-a3b2-4a75-beb8-25e793a12df0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:00.234855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e64a4e-a3b2-4a75-beb8-25e793a12df0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:00.242933 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:00.273842 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809710aa-b3dd-4a60-a371-2e2afb332270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:00.275010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809710aa-b3dd-4a60-a371-2e2afb332270 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:00.275125 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e3d4d-a6c1-4940-9c53-c870d0cfb996 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:00.275594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809710aa-b3dd-4a60-a371-2e2afb332270 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:00.276078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809710aa-b3dd-4a60-a371-2e2afb332270 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:00.285212 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809710aa-b3dd-4a60-a371-2e2afb332270 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:00.287077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809710aa-b3dd-4a60-a371-2e2afb332270 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-809710aa-b3dd-4a60-a371-2e2afb332270 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:00.303889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:00.304975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:00.305924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:00.305924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:00.305924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:00.305924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:00.305924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:00.305924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:00.305924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:00.305924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:00.308853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 381/775] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:00.315961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809710aa-b3dd-4a60-a371-2e2afb332270 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:00.369360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489366d4-62e3-47c2-9fa4-9233e91b6f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:00.370080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489366d4-62e3-47c2-9fa4-9233e91b6f59 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:00.370369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489366d4-62e3-47c2-9fa4-9233e91b6f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:00.370650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489366d4-62e3-47c2-9fa4-9233e91b6f59 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:00.371068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:00.371377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489366d4-62e3-47c2-9fa4-9233e91b6f59 None admin] RBAC: Adding query filter params (type=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:00.371688 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489366d4-62e3-47c2-9fa4-9233e91b6f59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:00.373102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489366d4-62e3-47c2-9fa4-9233e91b6f59 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:00.386142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 395/776] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:51:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:00.399108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489366d4-62e3-47c2-9fa4-9233e91b6f59 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:00.435709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809710aa-b3dd-4a60-a371-2e2afb332270 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:00.489913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15656f03-2d6c-491a-898a-2f0baf6430a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:00.491621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15656f03-2d6c-491a-898a-2f0baf6430a8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:00.492308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15656f03-2d6c-491a-898a-2f0baf6430a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:00.492994 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15656f03-2d6c-491a-898a-2f0baf6430a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:00.498433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15656f03-2d6c-491a-898a-2f0baf6430a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:00.499246 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15656f03-2d6c-491a-898a-2f0baf6430a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:00.502456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15656f03-2d6c-491a-898a-2f0baf6430a8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:00.543162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 382/777] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:51:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:00.553926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232d6f4-008d-480a-b876-b5898126530d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:00.554362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232d6f4-008d-480a-b876-b5898126530d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:00.554547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232d6f4-008d-480a-b876-b5898126530d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:00.557293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 383/778] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:00.558224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232d6f4-008d-480a-b876-b5898126530d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:00.592760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489366d4-62e3-47c2-9fa4-9233e91b6f59 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:00.629973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2acdfaa-583c-426f-861f-2a4157d3859e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:00.630665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acdfaa-583c-426f-861f-2a4157d3859e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:00.631003 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acdfaa-583c-426f-861f-2a4157d3859e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:00.631270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acdfaa-583c-426f-861f-2a4157d3859e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:00.632315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2acdfaa-583c-426f-861f-2a4157d3859e None admin] RBAC: Adding query filter params (interface=public, service_id=7d85eeabfa4f4bbeb069ccd1c132a6cb, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:00.632642 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2acdfaa-583c-426f-861f-2a4157d3859e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:00.634266 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2acdfaa-583c-426f-861f-2a4157d3859e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:00.643845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 396/779] 10.222.0.43 () {56 vars in 1251 bytes} [Mon Jul 4 07:51:00 2022] GET /identity/v3/endpoints?service_id=7d85eeabfa4f4bbeb069ccd1c132a6cb&interface=public®ion_id=RegionOne => generated 194 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:00.645391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c75405-f8ba-4a66-99c0-4cbcebc963fc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:00.645776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c75405-f8ba-4a66-99c0-4cbcebc963fc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:00.646070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c75405-f8ba-4a66-99c0-4cbcebc963fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:00.664861 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-62c75405-f8ba-4a66-99c0-4cbcebc963fc None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:00.666221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-62c75405-f8ba-4a66-99c0-4cbcebc963fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:00.679823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:00.698065 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:00.708209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232d6f4-008d-480a-b876-b5898126530d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:00.752349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007adf3-2b41-4aa8-94ae-ebf8c6eddbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:00.753317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007adf3-2b41-4aa8-94ae-ebf8c6eddbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:00.753775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007adf3-2b41-4aa8-94ae-ebf8c6eddbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:00.754538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007adf3-2b41-4aa8-94ae-ebf8c6eddbb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:00.761643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 397/780] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:00.761643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c75405-f8ba-4a66-99c0-4cbcebc963fc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:00.762247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007adf3-2b41-4aa8-94ae-ebf8c6eddbb7 None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=80a45a063c0e4f1e9b2156fb2f4cd8b8) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:00.762779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007adf3-2b41-4aa8-94ae-ebf8c6eddbb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:00.764308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007adf3-2b41-4aa8-94ae-ebf8c6eddbb7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:00.778334 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 384/781] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:51:00 2022] GET /identity/v3/role_assignments?user.id=80a45a063c0e4f1e9b2156fb2f4cd8b8&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:00.782989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007adf3-2b41-4aa8-94ae-ebf8c6eddbb7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:00.933742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007adf3-2b41-4aa8-94ae-ebf8c6eddbb7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:00.967410 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2805eb-e0d0-4a44-a13d-cee34dd6005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:00.968224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2805eb-e0d0-4a44-a13d-cee34dd6005c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:00.968508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2805eb-e0d0-4a44-a13d-cee34dd6005c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:00.968781 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2805eb-e0d0-4a44-a13d-cee34dd6005c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:00.978136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c75405-f8ba-4a66-99c0-4cbcebc963fc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:00.980407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2805eb-e0d0-4a44-a13d-cee34dd6005c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8b2805eb-e0d0-4a44-a13d-cee34dd6005c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:00.988147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:00.989242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:00.990160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 385/782] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:51:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:01.007076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2805eb-e0d0-4a44-a13d-cee34dd6005c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:01.049040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba98ac-1783-467b-9407-f5cc3a6f5bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:01.050047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:01.050887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba98ac-1783-467b-9407-f5cc3a6f5bfd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:01.051418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba98ac-1783-467b-9407-f5cc3a6f5bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:01.051773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba98ac-1783-467b-9407-f5cc3a6f5bfd None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:01.052471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba98ac-1783-467b-9407-f5cc3a6f5bfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:01.055012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba98ac-1783-467b-9407-f5cc3a6f5bfd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:01.061084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 398/783] 10.222.0.43 () {56 vars in 1087 bytes} [Mon Jul 4 07:51:00 2022] GET /identity/v3/services => generated 772 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:01.203391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2805eb-e0d0-4a44-a13d-cee34dd6005c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:01.255893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a3bca-52e8-40df-a094-8dc78840843b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:01.256671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a3bca-52e8-40df-a094-8dc78840843b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:01.256993 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a3bca-52e8-40df-a094-8dc78840843b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:01.257279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a3bca-52e8-40df-a094-8dc78840843b None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:01.258070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a3bca-52e8-40df-a094-8dc78840843b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:01.258374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a3bca-52e8-40df-a094-8dc78840843b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:01.259772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a3bca-52e8-40df-a094-8dc78840843b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:01.274266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 386/784] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:51:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:01.283352 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba98ac-1783-467b-9407-f5cc3a6f5bfd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:01.295666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:01.410758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c1255-57b1-41c5-9332-8960fa485b34 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:01.411442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c1255-57b1-41c5-9332-8960fa485b34 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:01.411938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c1255-57b1-41c5-9332-8960fa485b34 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:01.412931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba98ac-1783-467b-9407-f5cc3a6f5bfd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:01.414358 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 387/785] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:01.421781 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2362cf5-0fe0-4bb8-9d9e-f1405f7d072b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:01.422407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2362cf5-0fe0-4bb8-9d9e-f1405f7d072b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:01.422891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2362cf5-0fe0-4bb8-9d9e-f1405f7d072b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:01.442203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a2362cf5-0fe0-4bb8-9d9e-f1405f7d072b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:01.443910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a2362cf5-0fe0-4bb8-9d9e-f1405f7d072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:01.452946 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235dfed-05c2-4bed-8610-c429f6c649a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:01.454245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235dfed-05c2-4bed-8610-c429f6c649a8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:01.454758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235dfed-05c2-4bed-8610-c429f6c649a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:01.455240 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235dfed-05c2-4bed-8610-c429f6c649a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:01.467431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235dfed-05c2-4bed-8610-c429f6c649a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:01.469370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235dfed-05c2-4bed-8610-c429f6c649a8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9235dfed-05c2-4bed-8610-c429f6c649a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:01.491991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:01.493011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:01.494311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:01.494311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:01.494311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:01.494311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:01.494311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 399/786] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:01.499622 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235dfed-05c2-4bed-8610-c429f6c649a8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:01.542332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 388/787] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:01.582892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2362cf5-0fe0-4bb8-9d9e-f1405f7d072b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:01.599756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:01.609674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:01.649463 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235dfed-05c2-4bed-8610-c429f6c649a8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:01.695401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a480fa-ccb0-43f4-bc44-291b1bc038fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:01.696076 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a480fa-ccb0-43f4-bc44-291b1bc038fc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:01.696356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a480fa-ccb0-43f4-bc44-291b1bc038fc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:01.696628 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a480fa-ccb0-43f4-bc44-291b1bc038fc None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:01.697361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a480fa-ccb0-43f4-bc44-291b1bc038fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:01.697683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a480fa-ccb0-43f4-bc44-291b1bc038fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:01.699082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a480fa-ccb0-43f4-bc44-291b1bc038fc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:01.726801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 400/788] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:01.732737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a480fa-ccb0-43f4-bc44-291b1bc038fc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:01.772464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2362cf5-0fe0-4bb8-9d9e-f1405f7d072b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:01.835892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac72f9-2cf4-4ccc-962d-c45716187947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:01.836703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac72f9-2cf4-4ccc-962d-c45716187947 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:01.836977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac72f9-2cf4-4ccc-962d-c45716187947 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:01.837249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac72f9-2cf4-4ccc-962d-c45716187947 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:01.865449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac72f9-2cf4-4ccc-962d-c45716187947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-deac72f9-2cf4-4ccc-962d-c45716187947 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:01.873233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:01.874222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:01.875522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 389/789] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:51:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:01.886566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac72f9-2cf4-4ccc-962d-c45716187947 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:01.926106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a480fa-ccb0-43f4-bc44-291b1bc038fc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:01.950836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:01.963577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbeed5f-f762-46d9-9a98-825466a1645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:01.964593 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbeed5f-f762-46d9-9a98-825466a1645f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:01.964874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbeed5f-f762-46d9-9a98-825466a1645f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:01.965184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbeed5f-f762-46d9-9a98-825466a1645f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:01.974644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbeed5f-f762-46d9-9a98-825466a1645f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:01.976412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbeed5f-f762-46d9-9a98-825466a1645f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9dbeed5f-f762-46d9-9a98-825466a1645f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:51:02.007157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:02.009648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:02.011991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:02.011991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:02.011991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:51:02.011991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:51:02.011991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:51:02.011991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:51:02.011991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:51:02.011991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:51:02.011991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 07:51:02.011991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:02.011991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 401/790] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:51:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:02.016273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbeed5f-f762-46d9-9a98-825466a1645f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:02.038321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:02.135721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac72f9-2cf4-4ccc-962d-c45716187947 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:02.149054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbeed5f-f762-46d9-9a98-825466a1645f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:02.186182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89da01-4506-47e3-bb2e-3bf99689fd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:02.186957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89da01-4506-47e3-bb2e-3bf99689fd43 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:02.187238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89da01-4506-47e3-bb2e-3bf99689fd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:02.187509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89da01-4506-47e3-bb2e-3bf99689fd43 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:02.188360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89da01-4506-47e3-bb2e-3bf99689fd43 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:02.188665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89da01-4506-47e3-bb2e-3bf99689fd43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:02.190179 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89da01-4506-47e3-bb2e-3bf99689fd43 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:02.211184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 402/791] 10.222.0.43 () {56 vars in 1135 bytes} [Mon Jul 4 07:51:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:02.216448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89da01-4506-47e3-bb2e-3bf99689fd43 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:02.233178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57807e17-b483-466d-8c53-dca549cc45e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:02.233907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57807e17-b483-466d-8c53-dca549cc45e9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:02.234199 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57807e17-b483-466d-8c53-dca549cc45e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:02.234480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57807e17-b483-466d-8c53-dca549cc45e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:02.235240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57807e17-b483-466d-8c53-dca549cc45e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:02.235551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57807e17-b483-466d-8c53-dca549cc45e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:02.237086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57807e17-b483-466d-8c53-dca549cc45e9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:02.247510 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 390/792] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:51:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:02.249018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21561832-ebf0-4dce-ac1a-9a4f432b1261 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:02.249316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21561832-ebf0-4dce-ac1a-9a4f432b1261 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:02.249601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21561832-ebf0-4dce-ac1a-9a4f432b1261 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:02.251138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 391/793] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:02.254130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21561832-ebf0-4dce-ac1a-9a4f432b1261 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:02.264727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:02.333022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89da01-4506-47e3-bb2e-3bf99689fd43 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:02.372161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378f75c-f9c3-426b-a1e3-d28b3bb984c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:02.373348 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378f75c-f9c3-426b-a1e3-d28b3bb984c6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:02.373910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378f75c-f9c3-426b-a1e3-d28b3bb984c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:02.374359 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378f75c-f9c3-426b-a1e3-d28b3bb984c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:02.386150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378f75c-f9c3-426b-a1e3-d28b3bb984c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:02.387100 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21561832-ebf0-4dce-ac1a-9a4f432b1261 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:02.388202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378f75c-f9c3-426b-a1e3-d28b3bb984c6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7378f75c-f9c3-426b-a1e3-d28b3bb984c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:02.408934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:02.409939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:02.410905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:02.410905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:02.410905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:02.410905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:02.417856 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 403/794] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:02.419746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe23283-2b16-490b-a092-dd96ec5d9c5d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:02.420304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe23283-2b16-490b-a092-dd96ec5d9c5d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:02.420782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe23283-2b16-490b-a092-dd96ec5d9c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:02.426971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078413c8-1673-4c03-8097-4c0f51cc1bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:02.429659 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078413c8-1673-4c03-8097-4c0f51cc1bbd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:02.429953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078413c8-1673-4c03-8097-4c0f51cc1bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:02.430238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078413c8-1673-4c03-8097-4c0f51cc1bbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:02.432405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:02.441593 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-afe23283-2b16-490b-a092-dd96ec5d9c5d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:02.443145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-afe23283-2b16-490b-a092-dd96ec5d9c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:02.444737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078413c8-1673-4c03-8097-4c0f51cc1bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:02.446642 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078413c8-1673-4c03-8097-4c0f51cc1bbd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-078413c8-1673-4c03-8097-4c0f51cc1bbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:02.472394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:02.473408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:02.474364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:02.474364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:02.474364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:02.474364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:02.477700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 392/795] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:02.479619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078413c8-1673-4c03-8097-4c0f51cc1bbd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:02.525152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 404/796] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:02.526431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe23283-2b16-490b-a092-dd96ec5d9c5d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:02.550036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:02.604588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078413c8-1673-4c03-8097-4c0f51cc1bbd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:02.648319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5f037-a8cc-4fe6-8beb-3e447e1a5b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:02.650042 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5f037-a8cc-4fe6-8beb-3e447e1a5b22 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:02.650515 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5f037-a8cc-4fe6-8beb-3e447e1a5b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:02.651006 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5f037-a8cc-4fe6-8beb-3e447e1a5b22 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:02.652569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5f037-a8cc-4fe6-8beb-3e447e1a5b22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:02.653200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5f037-a8cc-4fe6-8beb-3e447e1a5b22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:02.655205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5f037-a8cc-4fe6-8beb-3e447e1a5b22 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:02.658343 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe23283-2b16-490b-a092-dd96ec5d9c5d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:02.675225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 393/797] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:02.676237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5f037-a8cc-4fe6-8beb-3e447e1a5b22 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:02.705969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d6b45-8e27-4639-97a6-8bf32b421980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:02.707669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d6b45-8e27-4639-97a6-8bf32b421980 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:02.708145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d6b45-8e27-4639-97a6-8bf32b421980 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:02.708836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d6b45-8e27-4639-97a6-8bf32b421980 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:02.710600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d6b45-8e27-4639-97a6-8bf32b421980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:02.711538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d6b45-8e27-4639-97a6-8bf32b421980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:02.713421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d6b45-8e27-4639-97a6-8bf32b421980 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:02.727816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:02.731107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 405/798] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:02.732302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d6b45-8e27-4639-97a6-8bf32b421980 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:02.790522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5f037-a8cc-4fe6-8beb-3e447e1a5b22 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:02.828072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd1337-dc04-4fb8-89ec-c7368344cdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:02.829026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd1337-dc04-4fb8-89ec-c7368344cdfa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:02.829381 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd1337-dc04-4fb8-89ec-c7368344cdfa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:02.829848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd1337-dc04-4fb8-89ec-c7368344cdfa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:02.830581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd1337-dc04-4fb8-89ec-c7368344cdfa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:02.832140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd1337-dc04-4fb8-89ec-c7368344cdfa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bbfd1337-dc04-4fb8-89ec-c7368344cdfa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:02.846437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:02.847475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:02.848347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 394/799] 10.222.0.43 () {56 vars in 1129 bytes} [Mon Jul 4 07:51:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:02.849224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd1337-dc04-4fb8-89ec-c7368344cdfa None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:02.851856 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:02.883017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d6b45-8e27-4639-97a6-8bf32b421980 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:02.923590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a98a1b-7636-499d-8f61-f80544f25332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:02.924474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a98a1b-7636-499d-8f61-f80544f25332 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:02.924749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a98a1b-7636-499d-8f61-f80544f25332 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:02.925018 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a98a1b-7636-499d-8f61-f80544f25332 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:02.933576 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a98a1b-7636-499d-8f61-f80544f25332 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:02.935173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a98a1b-7636-499d-8f61-f80544f25332 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-20a98a1b-7636-499d-8f61-f80544f25332 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:02.951314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:02.952318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:02.953249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:02.953249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:02.953249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:02.953249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:02.953249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:02.953249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:02.953249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:02.953249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:02.957561 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 406/800] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:02.958652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a98a1b-7636-499d-8f61-f80544f25332 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:03.070550 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a98a1b-7636-499d-8f61-f80544f25332 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:03.094854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd1337-dc04-4fb8-89ec-c7368344cdfa None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:03.108348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:03.113104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c8efe-f170-48f4-acfb-ca4631df2ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:03.113809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c8efe-f170-48f4-acfb-ca4631df2ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:03.114896 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c8efe-f170-48f4-acfb-ca4631df2ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:03.115274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c8efe-f170-48f4-acfb-ca4631df2ab9 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:03.116216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c8efe-f170-48f4-acfb-ca4631df2ab9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:03.116471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c8efe-f170-48f4-acfb-ca4631df2ab9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:03.118256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c8efe-f170-48f4-acfb-ca4631df2ab9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:03.124064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 407/801] 10.222.0.43 () {56 vars in 1124 bytes} [Mon Jul 4 07:51:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:03.125085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c8efe-f170-48f4-acfb-ca4631df2ab9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:03.134684 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:03.138395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f8e35-f7f3-4cd8-a394-d6b5e9927023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:03.139430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f8e35-f7f3-4cd8-a394-d6b5e9927023 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:03.139855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f8e35-f7f3-4cd8-a394-d6b5e9927023 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:03.140270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f8e35-f7f3-4cd8-a394-d6b5e9927023 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:03.148965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f8e35-f7f3-4cd8-a394-d6b5e9927023 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:03.150787 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f8e35-f7f3-4cd8-a394-d6b5e9927023 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b56f8e35-f7f3-4cd8-a394-d6b5e9927023 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:51:03.166500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:03.167534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:03.168522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:03.168522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:03.168522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:51:03.168522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:51:03.168522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:51:03.168522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:51:03.168522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:51:03.168522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:51:03.168522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:51:03.168522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:03.174723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 395/802] 10.222.0.43 () {56 vars in 1093 bytes} [Mon Jul 4 07:51:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:03.174723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f8e35-f7f3-4cd8-a394-d6b5e9927023 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:03.241680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c8efe-f170-48f4-acfb-ca4631df2ab9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:03.278066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859df0c-1bc3-48f6-8cba-0421478b7300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:03.278741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859df0c-1bc3-48f6-8cba-0421478b7300 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:03.279022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859df0c-1bc3-48f6-8cba-0421478b7300 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:03.279296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859df0c-1bc3-48f6-8cba-0421478b7300 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:03.280087 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859df0c-1bc3-48f6-8cba-0421478b7300 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:03.280400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859df0c-1bc3-48f6-8cba-0421478b7300 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:03.281863 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859df0c-1bc3-48f6-8cba-0421478b7300 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:03.285460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f8e35-f7f3-4cd8-a394-d6b5e9927023 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:03.294084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 408/803] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:51:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:03.295473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e06d86-ca69-46db-8537-d782c6218a93 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:03.295760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e06d86-ca69-46db-8537-d782c6218a93 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:03.296035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e06d86-ca69-46db-8537-d782c6218a93 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:03.297515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 409/804] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:03.302527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e06d86-ca69-46db-8537-d782c6218a93 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:03.327769 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d75ce-f122-4665-bc95-7b44057c3bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:03.328770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d75ce-f122-4665-bc95-7b44057c3bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:03.329324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d75ce-f122-4665-bc95-7b44057c3bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:03.329594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d75ce-f122-4665-bc95-7b44057c3bf8 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:03.330330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d75ce-f122-4665-bc95-7b44057c3bf8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:03.330631 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d75ce-f122-4665-bc95-7b44057c3bf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:03.332139 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d75ce-f122-4665-bc95-7b44057c3bf8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:03.343507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 396/805] 10.222.0.43 () {56 vars in 1124 bytes} [Mon Jul 4 07:51:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:03.344199 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d75ce-f122-4665-bc95-7b44057c3bf8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:03.373121 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:03.381320 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:03.418580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e06d86-ca69-46db-8537-d782c6218a93 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:03.460886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39989b30-6cae-4742-b9b5-72151f58a443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:03.461578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39989b30-6cae-4742-b9b5-72151f58a443 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:03.461906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39989b30-6cae-4742-b9b5-72151f58a443 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:03.462186 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39989b30-6cae-4742-b9b5-72151f58a443 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:03.462308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d75ce-f122-4665-bc95-7b44057c3bf8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:03.463036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39989b30-6cae-4742-b9b5-72151f58a443 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:03.463348 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39989b30-6cae-4742-b9b5-72151f58a443 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:03.465491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39989b30-6cae-4742-b9b5-72151f58a443 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:03.489445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 410/806] 10.222.0.43 () {56 vars in 1133 bytes} [Mon Jul 4 07:51:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:03.489445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a3847-45a3-47cf-8d92-c18bceb0e265 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:03.489445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a3847-45a3-47cf-8d92-c18bceb0e265 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:03.489445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a3847-45a3-47cf-8d92-c18bceb0e265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:03.502801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62e306-1562-48d3-8746-3e65470749da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:03.503720 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62e306-1562-48d3-8746-3e65470749da None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:03.504004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62e306-1562-48d3-8746-3e65470749da None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:03.504624 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62e306-1562-48d3-8746-3e65470749da None admin] PATH_INFO: `/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/6763884b51204c488ca8c7e452e3a29e/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:03.507078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62e306-1562-48d3-8746-3e65470749da None admin] RBAC: Authorizing `identity:create_grant(project_id=1f2d85c9e3d14bdabfb63fe505d83714, user_id=6763884b51204c488ca8c7e452e3a29e, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:03.508453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ff4a3847-45a3-47cf-8d92-c18bceb0e265 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:03.508788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62e306-1562-48d3-8746-3e65470749da None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:03.509569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ff4a3847-45a3-47cf-8d92-c18bceb0e265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:03.537667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 397/807] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:51:03 2022] PUT /identity/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/6763884b51204c488ca8c7e452e3a29e/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:51:03.537667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62e306-1562-48d3-8746-3e65470749da None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:03.607192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 411/808] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:03.608004 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a3847-45a3-47cf-8d92-c18bceb0e265 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:03.674313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:03.682849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62e306-1562-48d3-8746-3e65470749da None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:03.684410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:03.728015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d72a58-542d-4af1-a1ac-f5551ea61561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:03.728701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d72a58-542d-4af1-a1ac-f5551ea61561 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:03.728991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d72a58-542d-4af1-a1ac-f5551ea61561 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:03.729271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d72a58-542d-4af1-a1ac-f5551ea61561 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:03.730135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d72a58-542d-4af1-a1ac-f5551ea61561 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:03.731540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d72a58-542d-4af1-a1ac-f5551ea61561 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:03.745049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a3847-45a3-47cf-8d92-c18bceb0e265 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:03.758673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 398/809] 10.222.0.43 () {60 vars in 1144 bytes} [Mon Jul 4 07:51:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:03.760258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599907aa-fbb6-4a5d-b8df-1daa3d3d1d2a None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:03.760697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599907aa-fbb6-4a5d-b8df-1daa3d3d1d2a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:03.761120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599907aa-fbb6-4a5d-b8df-1daa3d3d1d2a None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:03.762840 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 399/810] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:03.767772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599907aa-fbb6-4a5d-b8df-1daa3d3d1d2a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:03.817783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b403af-5de4-49bc-915a-83021c8f2eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:03.818586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b403af-5de4-49bc-915a-83021c8f2eaf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:03.818945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b403af-5de4-49bc-915a-83021c8f2eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:03.819656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b403af-5de4-49bc-915a-83021c8f2eaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:03.917562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b403af-5de4-49bc-915a-83021c8f2eaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:03.919125 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b403af-5de4-49bc-915a-83021c8f2eaf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:03.928881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599907aa-fbb6-4a5d-b8df-1daa3d3d1d2a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:03.984694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d601d9e-8c41-4977-969b-5f902d777926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:03.986241 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d601d9e-8c41-4977-969b-5f902d777926 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:03.987007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d601d9e-8c41-4977-969b-5f902d777926 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:03.987829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d601d9e-8c41-4977-969b-5f902d777926 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:03.994212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:04.009319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d601d9e-8c41-4977-969b-5f902d777926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5d601d9e-8c41-4977-969b-5f902d777926 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:04.017118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:04.018969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 400/811] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:51:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:04.020795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de912f2-cd0e-4d9e-b023-1ec8244b0457 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:04.023168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de912f2-cd0e-4d9e-b023-1ec8244b0457 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:04.023168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de912f2-cd0e-4d9e-b023-1ec8244b0457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:04.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5de912f2-cd0e-4d9e-b023-1ec8244b0457 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:04.041535 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5de912f2-cd0e-4d9e-b023-1ec8244b0457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:04.044838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:04.067055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 412/812] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:51:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:51:04.067055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b403af-5de4-49bc-915a-83021c8f2eaf None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:04.149401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 401/813] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:04.151153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de912f2-cd0e-4d9e-b023-1ec8244b0457 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:04.207587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b403af-5de4-49bc-915a-83021c8f2eaf None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:04.259082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b6423-022d-461a-8a43-92903e2c2d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:04.260065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b6423-022d-461a-8a43-92903e2c2d08 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:04.260496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b6423-022d-461a-8a43-92903e2c2d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:04.260964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b6423-022d-461a-8a43-92903e2c2d08 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:04.261908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b6423-022d-461a-8a43-92903e2c2d08 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:04.262427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b6423-022d-461a-8a43-92903e2c2d08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:04.264060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b6423-022d-461a-8a43-92903e2c2d08 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:04.278644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 413/814] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:51:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:04.279918 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b6423-022d-461a-8a43-92903e2c2d08 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:04.294726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:04.308986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de912f2-cd0e-4d9e-b023-1ec8244b0457 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:04.353745 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517af8b-08ae-46fd-8f04-cf52a5a4a673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:04.354556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517af8b-08ae-46fd-8f04-cf52a5a4a673 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:04.354841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517af8b-08ae-46fd-8f04-cf52a5a4a673 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:04.355117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517af8b-08ae-46fd-8f04-cf52a5a4a673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:04.366399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:04.370891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517af8b-08ae-46fd-8f04-cf52a5a4a673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:04.372676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517af8b-08ae-46fd-8f04-cf52a5a4a673 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0517af8b-08ae-46fd-8f04-cf52a5a4a673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:04.396980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:04.399399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:04.401409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:04.401409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:04.401409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:04.401409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:04.401409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 402/815] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:04.405989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517af8b-08ae-46fd-8f04-cf52a5a4a673 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:04.436896 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b6423-022d-461a-8a43-92903e2c2d08 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:04.474777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a2dce-d37f-42f8-8f01-e1c228fd3aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:04.476289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a2dce-d37f-42f8-8f01-e1c228fd3aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:04.476720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a2dce-d37f-42f8-8f01-e1c228fd3aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:04.477148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a2dce-d37f-42f8-8f01-e1c228fd3aa1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:04.477982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a2dce-d37f-42f8-8f01-e1c228fd3aa1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:04.479771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a2dce-d37f-42f8-8f01-e1c228fd3aa1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b03a2dce-d37f-42f8-8f01-e1c228fd3aa1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:04.493000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:04.494324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 07:51:04.495398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:04.498202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 414/816] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:04 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:04.499373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a2dce-d37f-42f8-8f01-e1c228fd3aa1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:04.522402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517af8b-08ae-46fd-8f04-cf52a5a4a673 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:04.560817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1b9a6-7836-4802-9023-ab906cb013f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:04.561740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1b9a6-7836-4802-9023-ab906cb013f2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:04.561999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1b9a6-7836-4802-9023-ab906cb013f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:04.562345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1b9a6-7836-4802-9023-ab906cb013f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:04.574041 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1b9a6-7836-4802-9023-ab906cb013f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:04.575551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1b9a6-7836-4802-9023-ab906cb013f2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-6fb1b9a6-7836-4802-9023-ab906cb013f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:04.596393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:04.597577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:04.598591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:04.598591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:04.598591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:04.598591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:04.602927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 403/817] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:04.603958 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1b9a6-7836-4802-9023-ab906cb013f2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:04.607862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:04.640350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a2dce-d37f-42f8-8f01-e1c228fd3aa1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:04.670946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:04.684513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903126c-dad9-4f6f-b94b-6b88d1c30a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:04.685382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903126c-dad9-4f6f-b94b-6b88d1c30a81 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:04.685917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903126c-dad9-4f6f-b94b-6b88d1c30a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:04.686303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903126c-dad9-4f6f-b94b-6b88d1c30a81 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:04.687267 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903126c-dad9-4f6f-b94b-6b88d1c30a81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:04.687670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903126c-dad9-4f6f-b94b-6b88d1c30a81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:04.689349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903126c-dad9-4f6f-b94b-6b88d1c30a81 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:04.704983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 415/818] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:04.706610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903126c-dad9-4f6f-b94b-6b88d1c30a81 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:04.751780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1b9a6-7836-4802-9023-ab906cb013f2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:04.790365 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22379eea-b9bd-48af-bacd-8f02f0de57da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:04.791260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22379eea-b9bd-48af-bacd-8f02f0de57da None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:04.791639 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22379eea-b9bd-48af-bacd-8f02f0de57da None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:04.792040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22379eea-b9bd-48af-bacd-8f02f0de57da None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:04.792855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22379eea-b9bd-48af-bacd-8f02f0de57da None admin] RBAC: Adding query filter params (name=network) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:04.793265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22379eea-b9bd-48af-bacd-8f02f0de57da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:04.794827 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22379eea-b9bd-48af-bacd-8f02f0de57da None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:04.800497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 404/819] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:51:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:04.801454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22379eea-b9bd-48af-bacd-8f02f0de57da None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:04.823902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903126c-dad9-4f6f-b94b-6b88d1c30a81 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:04.861771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc1c65-b27e-41f8-a92b-6d01211eedfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:04.862798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1c65-b27e-41f8-a92b-6d01211eedfb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:04.863251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1c65-b27e-41f8-a92b-6d01211eedfb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:04.863781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1c65-b27e-41f8-a92b-6d01211eedfb None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:04.864693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc1c65-b27e-41f8-a92b-6d01211eedfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:04.865170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc1c65-b27e-41f8-a92b-6d01211eedfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:04.867067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc1c65-b27e-41f8-a92b-6d01211eedfb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:04.880323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 416/820] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:04.881703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc1c65-b27e-41f8-a92b-6d01211eedfb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:04.906831 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22379eea-b9bd-48af-bacd-8f02f0de57da None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:04.916960 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:04.920587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:04.943674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7a6e0-3327-4e40-a0d0-cbfc5c671bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:04.944775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7a6e0-3327-4e40-a0d0-cbfc5c671bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:04.945127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7a6e0-3327-4e40-a0d0-cbfc5c671bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:04.945583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7a6e0-3327-4e40-a0d0-cbfc5c671bd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:04.953207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7a6e0-3327-4e40-a0d0-cbfc5c671bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:04.954706 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7a6e0-3327-4e40-a0d0-cbfc5c671bd4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-d8f7a6e0-3327-4e40-a0d0-cbfc5c671bd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:04.969425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:04.970503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:04.971748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:04.971748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:04.971748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:04.971748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:04.971748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:04.971748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:04.971748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:04.971748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:04.971748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 405/821] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:04.972244 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7a6e0-3327-4e40-a0d0-cbfc5c671bd4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:04.997967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc1c65-b27e-41f8-a92b-6d01211eedfb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:05.039421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35c7fe-45ad-44a6-b541-453571489102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:05.040363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35c7fe-45ad-44a6-b541-453571489102 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:05.040715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35c7fe-45ad-44a6-b541-453571489102 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:05.041063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35c7fe-45ad-44a6-b541-453571489102 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:05.041911 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35c7fe-45ad-44a6-b541-453571489102 None admin] RBAC: Adding query filter params (type=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:05.042536 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35c7fe-45ad-44a6-b541-453571489102 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:05.044818 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35c7fe-45ad-44a6-b541-453571489102 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:05.057135 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 417/822] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:51:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:05.058305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35c7fe-45ad-44a6-b541-453571489102 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:05.104503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7a6e0-3327-4e40-a0d0-cbfc5c671bd4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:05.138564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:05.148489 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b8a85-5030-4a57-a0ac-403e80e3180e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:05.149428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b8a85-5030-4a57-a0ac-403e80e3180e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:05.149997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b8a85-5030-4a57-a0ac-403e80e3180e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:05.149997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b8a85-5030-4a57-a0ac-403e80e3180e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:05.158712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b8a85-5030-4a57-a0ac-403e80e3180e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:05.160299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b8a85-5030-4a57-a0ac-403e80e3180e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8b9b8a85-5030-4a57-a0ac-403e80e3180e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:51:05.175857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:05.176894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:05.178174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:05.178174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:05.178174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:51:05.178174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:51:05.178174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:51:05.178174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:51:05.178174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:51:05.178174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:51:05.178174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 07:51:05.178174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:05.178174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 406/823] 10.222.0.43 () {56 vars in 1117 bytes} [Mon Jul 4 07:51:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:05.179703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b8a85-5030-4a57-a0ac-403e80e3180e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:05.193652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35c7fe-45ad-44a6-b541-453571489102 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:05.210378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:05.235041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed39ac5-3ab7-468c-9a84-10d7e1ff27ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:05.236064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed39ac5-3ab7-468c-9a84-10d7e1ff27ba None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:05.236514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed39ac5-3ab7-468c-9a84-10d7e1ff27ba None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:05.236933 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed39ac5-3ab7-468c-9a84-10d7e1ff27ba None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:05.237907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed39ac5-3ab7-468c-9a84-10d7e1ff27ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:05.238378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed39ac5-3ab7-468c-9a84-10d7e1ff27ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:05.240263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed39ac5-3ab7-468c-9a84-10d7e1ff27ba None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:05.255438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 418/824] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:51:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:05.256676 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed39ac5-3ab7-468c-9a84-10d7e1ff27ba None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:05.285982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b8a85-5030-4a57-a0ac-403e80e3180e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:05.321726 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f58ab-375d-41cb-b24a-bae6d59b0430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:05.322414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f58ab-375d-41cb-b24a-bae6d59b0430 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:05.322829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f58ab-375d-41cb-b24a-bae6d59b0430 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:05.323184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f58ab-375d-41cb-b24a-bae6d59b0430 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:05.324028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f58ab-375d-41cb-b24a-bae6d59b0430 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:05.325737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f58ab-375d-41cb-b24a-bae6d59b0430 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:05.353767 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 407/825] 10.222.0.43 () {60 vars in 1144 bytes} [Mon Jul 4 07:51:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:05.355014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f58ab-375d-41cb-b24a-bae6d59b0430 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:05.376451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed39ac5-3ab7-468c-9a84-10d7e1ff27ba None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:05.398722 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:05.427685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14253645-e316-4b18-81e3-350431e57944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:05.428962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14253645-e316-4b18-81e3-350431e57944 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:05.429587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14253645-e316-4b18-81e3-350431e57944 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:05.430090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14253645-e316-4b18-81e3-350431e57944 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:05.431732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14253645-e316-4b18-81e3-350431e57944 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:05.432289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14253645-e316-4b18-81e3-350431e57944 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:05.434056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14253645-e316-4b18-81e3-350431e57944 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:05.457151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 419/826] 10.222.0.43 () {56 vars in 1149 bytes} [Mon Jul 4 07:51:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:05.466406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14253645-e316-4b18-81e3-350431e57944 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:05.526764 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:05.562579 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f58ab-375d-41cb-b24a-bae6d59b0430 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:05.607726 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a6579-0e88-4d11-868a-5f7099f2bbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:05.608319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a6579-0e88-4d11-868a-5f7099f2bbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:05.608606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a6579-0e88-4d11-868a-5f7099f2bbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:05.608885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a6579-0e88-4d11-868a-5f7099f2bbc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:05.609780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a6579-0e88-4d11-868a-5f7099f2bbc0 None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=15211e42ec9d4c1699d877f0b55ce918) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:05.610096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a6579-0e88-4d11-868a-5f7099f2bbc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:05.611696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a6579-0e88-4d11-868a-5f7099f2bbc0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:05.619474 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 408/827] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:51:05 2022] GET /identity/v3/role_assignments?user.id=15211e42ec9d4c1699d877f0b55ce918&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:05.646086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14253645-e316-4b18-81e3-350431e57944 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:05.695204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f3705-3797-4cbb-8731-2e91b79ea1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:05.696418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f3705-3797-4cbb-8731-2e91b79ea1db None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:05.696901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f3705-3797-4cbb-8731-2e91b79ea1db None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:05.697395 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f3705-3797-4cbb-8731-2e91b79ea1db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:05.710208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f3705-3797-4cbb-8731-2e91b79ea1db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:05.712273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f3705-3797-4cbb-8731-2e91b79ea1db None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e44f3705-3797-4cbb-8731-2e91b79ea1db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:05.733238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:05.734312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:05.735242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:05.735242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:05.735242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:05.735242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:05.737719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 420/828] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:05.743442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a6579-0e88-4d11-868a-5f7099f2bbc0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:05.777137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66effbb-67bc-41de-ace2-f332dcf867b7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:05.777500 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66effbb-67bc-41de-ace2-f332dcf867b7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:05.777705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66effbb-67bc-41de-ace2-f332dcf867b7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:05.779568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 421/829] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:05.786891 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb42bc-69a7-43ce-9c4f-c687a011a01e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:05.787244 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb42bc-69a7-43ce-9c4f-c687a011a01e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:05.787656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb42bc-69a7-43ce-9c4f-c687a011a01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:05.811014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-31eb42bc-69a7-43ce-9c4f-c687a011a01e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:05.812268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-31eb42bc-69a7-43ce-9c4f-c687a011a01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:05.834666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:05.875786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a6579-0e88-4d11-868a-5f7099f2bbc0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:05.896982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 422/830] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:05.921306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb42bc-69a7-43ce-9c4f-c687a011a01e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:05.923648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b37216-4e5d-44e2-840c-f4eb97067260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:05.924327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b37216-4e5d-44e2-840c-f4eb97067260 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:05.924600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b37216-4e5d-44e2-840c-f4eb97067260 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:05.924880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b37216-4e5d-44e2-840c-f4eb97067260 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:05.925645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b37216-4e5d-44e2-840c-f4eb97067260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:05.925956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b37216-4e5d-44e2-840c-f4eb97067260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:05.927348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b37216-4e5d-44e2-840c-f4eb97067260 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:05.942480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 409/831] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:05.947624 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b37216-4e5d-44e2-840c-f4eb97067260 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:05.986006 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:06.035703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb42bc-69a7-43ce-9c4f-c687a011a01e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:06.073647 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580be32-256e-4638-9e42-23e15dee60c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:06.074464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580be32-256e-4638-9e42-23e15dee60c0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:06.074744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580be32-256e-4638-9e42-23e15dee60c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:06.075018 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580be32-256e-4638-9e42-23e15dee60c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:06.082753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b37216-4e5d-44e2-840c-f4eb97067260 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:06.086908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580be32-256e-4638-9e42-23e15dee60c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c580be32-256e-4638-9e42-23e15dee60c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:06.094751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:06.095922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:06.096847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 423/832] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:51:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:06.097688 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0b1fd-573a-4469-8520-17847ed5c9ca None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:06.097688 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0b1fd-573a-4469-8520-17847ed5c9ca None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:06.097957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0b1fd-573a-4469-8520-17847ed5c9ca None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:06.099424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 424/833] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:06.103910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0b1fd-573a-4469-8520-17847ed5c9ca None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:06.124596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:06.134265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773dba24-384a-4b7e-92bf-6f477b7c945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:06.135002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773dba24-384a-4b7e-92bf-6f477b7c945e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:06.135281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773dba24-384a-4b7e-92bf-6f477b7c945e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:06.135550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773dba24-384a-4b7e-92bf-6f477b7c945e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:06.145926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773dba24-384a-4b7e-92bf-6f477b7c945e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:06.148306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773dba24-384a-4b7e-92bf-6f477b7c945e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-773dba24-384a-4b7e-92bf-6f477b7c945e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:06.171386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:06.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:06.173916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:06.173916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:06.173916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:06.173916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:06.173916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:06.173916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:06.173916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:06.173916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:06.178409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 410/834] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:06.179890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051532a-e46e-4d51-b1ce-7065c87c0678 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:06.180195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051532a-e46e-4d51-b1ce-7065c87c0678 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:06.180458 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051532a-e46e-4d51-b1ce-7065c87c0678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:06.198471 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2051532a-e46e-4d51-b1ce-7065c87c0678 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:06.199700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2051532a-e46e-4d51-b1ce-7065c87c0678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:06.230615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0b1fd-573a-4469-8520-17847ed5c9ca None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:06.266956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:06.269936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb3d2d-04a1-4bd1-886f-96f717240d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:06.270627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb3d2d-04a1-4bd1-886f-96f717240d83 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:06.270957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb3d2d-04a1-4bd1-886f-96f717240d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:06.271233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb3d2d-04a1-4bd1-886f-96f717240d83 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:06.271990 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb3d2d-04a1-4bd1-886f-96f717240d83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:06.272323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb3d2d-04a1-4bd1-886f-96f717240d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:06.273670 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 411/835] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:06.274438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb3d2d-04a1-4bd1-886f-96f717240d83 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:06.282886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051532a-e46e-4d51-b1ce-7065c87c0678 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:06.288955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 425/836] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:06.289916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb3d2d-04a1-4bd1-886f-96f717240d83 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:06.372910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:06.397729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051532a-e46e-4d51-b1ce-7065c87c0678 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:06.399052 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb3d2d-04a1-4bd1-886f-96f717240d83 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:06.435262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487af51f-fa49-4d6a-b0a9-3c1a70ca3ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:06.435499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09d627-b676-4298-816b-d7e3cd44b3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:06.436070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09d627-b676-4298-816b-d7e3cd44b3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:06.436164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487af51f-fa49-4d6a-b0a9-3c1a70ca3ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:06.436402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09d627-b676-4298-816b-d7e3cd44b3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:06.436500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487af51f-fa49-4d6a-b0a9-3c1a70ca3ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:06.436678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09d627-b676-4298-816b-d7e3cd44b3e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:06.436772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487af51f-fa49-4d6a-b0a9-3c1a70ca3ec4 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:06.437483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487af51f-fa49-4d6a-b0a9-3c1a70ca3ec4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:06.437826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487af51f-fa49-4d6a-b0a9-3c1a70ca3ec4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:06.439320 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487af51f-fa49-4d6a-b0a9-3c1a70ca3ec4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:06.448162 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09d627-b676-4298-816b-d7e3cd44b3e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:06.449654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09d627-b676-4298-816b-d7e3cd44b3e1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:06.456803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 412/837] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:51:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:06.456803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487af51f-fa49-4d6a-b0a9-3c1a70ca3ec4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8e09d627-b676-4298-816b-d7e3cd44b3e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:06.473782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:06.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:06.475770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:06.475770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:06.475770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:06.475770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:06.475770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 426/838] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:06.477084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09d627-b676-4298-816b-d7e3cd44b3e1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:06.519099 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:06.562975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487af51f-fa49-4d6a-b0a9-3c1a70ca3ec4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:06.590887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09d627-b676-4298-816b-d7e3cd44b3e1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:06.599279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71ae2d-73ba-4b87-84f7-19ca15ef2fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:06.600100 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71ae2d-73ba-4b87-84f7-19ca15ef2fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:06.600416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71ae2d-73ba-4b87-84f7-19ca15ef2fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:06.600851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71ae2d-73ba-4b87-84f7-19ca15ef2fe0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:06.601536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71ae2d-73ba-4b87-84f7-19ca15ef2fe0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:06.602952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71ae2d-73ba-4b87-84f7-19ca15ef2fe0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-df71ae2d-73ba-4b87-84f7-19ca15ef2fe0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:06.615712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:06.618209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 413/839] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:51:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:06.619024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71ae2d-73ba-4b87-84f7-19ca15ef2fe0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:06.626176 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:06.636094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bf4f7-9ccf-444e-90b0-2c46dfe31b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:06.637030 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bf4f7-9ccf-444e-90b0-2c46dfe31b37 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:06.637360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bf4f7-9ccf-444e-90b0-2c46dfe31b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:06.637693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bf4f7-9ccf-444e-90b0-2c46dfe31b37 None admin] PATH_INFO: `/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/80a45a063c0e4f1e9b2156fb2f4cd8b8/roles/fd54d351b4ac49d7be6d0690309c00e7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:06.653560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bf4f7-9ccf-444e-90b0-2c46dfe31b37 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f2d85c9e3d14bdabfb63fe505d83714, user_id=80a45a063c0e4f1e9b2156fb2f4cd8b8, role_id=fd54d351b4ac49d7be6d0690309c00e7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:06.654842 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bf4f7-9ccf-444e-90b0-2c46dfe31b37 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:06.677711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 427/840] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:51:06 2022] PUT /identity/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/80a45a063c0e4f1e9b2156fb2f4cd8b8/roles/fd54d351b4ac49d7be6d0690309c00e7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:51:06.677711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bf4f7-9ccf-444e-90b0-2c46dfe31b37 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:06.763522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71ae2d-73ba-4b87-84f7-19ca15ef2fe0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:06.796018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:06.796300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bf4f7-9ccf-444e-90b0-2c46dfe31b37 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:06.814086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1d8b4-ea16-4c12-9b2b-b8e4fb6fecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:06.815778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1d8b4-ea16-4c12-9b2b-b8e4fb6fecb4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:06.816057 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1d8b4-ea16-4c12-9b2b-b8e4fb6fecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:06.816344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1d8b4-ea16-4c12-9b2b-b8e4fb6fecb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:06.817122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1d8b4-ea16-4c12-9b2b-b8e4fb6fecb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:06.817428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1d8b4-ea16-4c12-9b2b-b8e4fb6fecb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:06.818989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1d8b4-ea16-4c12-9b2b-b8e4fb6fecb4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:06.838039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 414/841] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:06.839084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1d8b4-ea16-4c12-9b2b-b8e4fb6fecb4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:06.845843 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd158348-3b4d-4a31-8c2f-8863bbe2b4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:06.846551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd158348-3b4d-4a31-8c2f-8863bbe2b4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:06.846855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd158348-3b4d-4a31-8c2f-8863bbe2b4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:06.850149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd158348-3b4d-4a31-8c2f-8863bbe2b4c2 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:06.851855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd158348-3b4d-4a31-8c2f-8863bbe2b4c2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:06.852187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd158348-3b4d-4a31-8c2f-8863bbe2b4c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:06.854920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd158348-3b4d-4a31-8c2f-8863bbe2b4c2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:06.861438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 428/842] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:51:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:06.867197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd158348-3b4d-4a31-8c2f-8863bbe2b4c2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:06.945398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:06.978785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1d8b4-ea16-4c12-9b2b-b8e4fb6fecb4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:07.016507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bba7e-b82f-4d29-bf51-961960c91e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:07.017759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bba7e-b82f-4d29-bf51-961960c91e44 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:07.018266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bba7e-b82f-4d29-bf51-961960c91e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:07.018763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bba7e-b82f-4d29-bf51-961960c91e44 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:07.022599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd158348-3b4d-4a31-8c2f-8863bbe2b4c2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:07.027906 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2bba7e-b82f-4d29-bf51-961960c91e44 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:07.029728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2bba7e-b82f-4d29-bf51-961960c91e44 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0f2bba7e-b82f-4d29-bf51-961960c91e44 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:51:07.048101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:07.049270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:07.050851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:07.050851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:07.050851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:51:07.050851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:51:07.050851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:51:07.050851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:51:07.050851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:51:07.050851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:51:07.050851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 07:51:07.050851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:07.058464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 415/843] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:51:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:07.060495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bba7e-b82f-4d29-bf51-961960c91e44 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:07.063162 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefe0c5-94a0-4fc9-82cd-fcdfe1bf3774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:07.063858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefe0c5-94a0-4fc9-82cd-fcdfe1bf3774 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:07.064122 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefe0c5-94a0-4fc9-82cd-fcdfe1bf3774 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:07.064399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefe0c5-94a0-4fc9-82cd-fcdfe1bf3774 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:07.065110 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefe0c5-94a0-4fc9-82cd-fcdfe1bf3774 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:07.065410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefe0c5-94a0-4fc9-82cd-fcdfe1bf3774 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:07.068218 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefe0c5-94a0-4fc9-82cd-fcdfe1bf3774 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:07.073799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 429/844] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:51:06 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:07.123868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:07.189242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bba7e-b82f-4d29-bf51-961960c91e44 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:07.228292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822a3d6-f166-4630-9c01-a488bfda2516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:07.228988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822a3d6-f166-4630-9c01-a488bfda2516 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:07.229266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822a3d6-f166-4630-9c01-a488bfda2516 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:07.229534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822a3d6-f166-4630-9c01-a488bfda2516 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:07.230419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822a3d6-f166-4630-9c01-a488bfda2516 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:07.230722 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822a3d6-f166-4630-9c01-a488bfda2516 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:07.232195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822a3d6-f166-4630-9c01-a488bfda2516 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:07.252494 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 416/845] 10.222.0.43 () {56 vars in 1135 bytes} [Mon Jul 4 07:51:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:07.259359 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefe0c5-94a0-4fc9-82cd-fcdfe1bf3774 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:07.399483 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefe0c5-94a0-4fc9-82cd-fcdfe1bf3774 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:07.409146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:07.441301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dc17d-e4ea-4a09-a4a3-0ff23c180045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:07.442003 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dc17d-e4ea-4a09-a4a3-0ff23c180045 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:07.442291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dc17d-e4ea-4a09-a4a3-0ff23c180045 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:07.442570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dc17d-e4ea-4a09-a4a3-0ff23c180045 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:07.523350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13dc17d-e4ea-4a09-a4a3-0ff23c180045 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:07.524766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13dc17d-e4ea-4a09-a4a3-0ff23c180045 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:07.631673 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 430/846] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:51:07 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:51:07.638841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822a3d6-f166-4630-9c01-a488bfda2516 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:07.746797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:07.750678 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822a3d6-f166-4630-9c01-a488bfda2516 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:07.789477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4397b-f7cc-4cfd-b1e7-0d3e4e37df38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:07.790280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4397b-f7cc-4cfd-b1e7-0d3e4e37df38 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:07.790607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4397b-f7cc-4cfd-b1e7-0d3e4e37df38 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:07.790890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4397b-f7cc-4cfd-b1e7-0d3e4e37df38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:07.806097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4397b-f7cc-4cfd-b1e7-0d3e4e37df38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:07.807585 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4397b-f7cc-4cfd-b1e7-0d3e4e37df38 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2aa4397b-f7cc-4cfd-b1e7-0d3e4e37df38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:07.827401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:07.828392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:07.829343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:07.829343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:07.829343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:07.829343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:07.833545 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 417/847] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:07.835170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dc17d-e4ea-4a09-a4a3-0ff23c180045 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:07.855326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:07.855580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbc648-e6b2-4ca2-a4ac-214fdce43e28 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:07.855900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbc648-e6b2-4ca2-a4ac-214fdce43e28 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:07.856274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbc648-e6b2-4ca2-a4ac-214fdce43e28 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:07.858151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 418/848] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:07.863530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216c47b-2ca2-4832-8d0c-18cb1afe5aea None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:07.863823 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216c47b-2ca2-4832-8d0c-18cb1afe5aea None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:07.864091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216c47b-2ca2-4832-8d0c-18cb1afe5aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:07.882190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e216c47b-2ca2-4832-8d0c-18cb1afe5aea None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:07.883390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e216c47b-2ca2-4832-8d0c-18cb1afe5aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:07.972152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dc17d-e4ea-4a09-a4a3-0ff23c180045 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:07.976763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 419/849] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:08.011217 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216c47b-2ca2-4832-8d0c-18cb1afe5aea None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:08.022830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7108a-b69c-41d8-a285-3919fc4032e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:08.023522 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7108a-b69c-41d8-a285-3919fc4032e1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:08.023798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7108a-b69c-41d8-a285-3919fc4032e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:08.024076 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7108a-b69c-41d8-a285-3919fc4032e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:08.024815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7108a-b69c-41d8-a285-3919fc4032e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:08.025123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7108a-b69c-41d8-a285-3919fc4032e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:08.026597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7108a-b69c-41d8-a285-3919fc4032e1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:08.039474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 431/850] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:08.046723 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7108a-b69c-41d8-a285-3919fc4032e1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:08.106087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:08.126389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:08.129851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216c47b-2ca2-4832-8d0c-18cb1afe5aea None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:08.167672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7108a-b69c-41d8-a285-3919fc4032e1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:08.170207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f435e-76c3-4f7d-8501-94d71786013c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:08.171378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f435e-76c3-4f7d-8501-94d71786013c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:08.171869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f435e-76c3-4f7d-8501-94d71786013c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:08.172344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f435e-76c3-4f7d-8501-94d71786013c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:08.173204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f435e-76c3-4f7d-8501-94d71786013c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:08.174929 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f435e-76c3-4f7d-8501-94d71786013c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e10f435e-76c3-4f7d-8501-94d71786013c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:08.189310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:08.190854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 07:51:08.191728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:08.195361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 420/851] 10.222.0.43 () {56 vars in 1132 bytes} [Mon Jul 4 07:51:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:08.201736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f435e-76c3-4f7d-8501-94d71786013c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:08.211788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302300c-3a37-40ad-9038-6a2b6264aaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:08.211788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302300c-3a37-40ad-9038-6a2b6264aaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:08.211788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302300c-3a37-40ad-9038-6a2b6264aaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:08.212130 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302300c-3a37-40ad-9038-6a2b6264aaa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:08.220525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302300c-3a37-40ad-9038-6a2b6264aaa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:08.222090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302300c-3a37-40ad-9038-6a2b6264aaa1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8302300c-3a37-40ad-9038-6a2b6264aaa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:08.249405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:08.250418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:08.251340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:08.251340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:08.251340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:08.251340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:08.251340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:08.251340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:08.251340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:08.251340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:08.253956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 432/852] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:08.262711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302300c-3a37-40ad-9038-6a2b6264aaa1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:08.333310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f435e-76c3-4f7d-8501-94d71786013c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:08.375825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579082db-ba9b-4771-9040-85945c662dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:08.376765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579082db-ba9b-4771-9040-85945c662dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:08.377052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579082db-ba9b-4771-9040-85945c662dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:08.377330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579082db-ba9b-4771-9040-85945c662dd7 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:08.378342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579082db-ba9b-4771-9040-85945c662dd7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:08.379033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579082db-ba9b-4771-9040-85945c662dd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:08.380801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579082db-ba9b-4771-9040-85945c662dd7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:08.388279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 421/853] 10.222.0.43 () {56 vars in 1126 bytes} [Mon Jul 4 07:51:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:08.394102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579082db-ba9b-4771-9040-85945c662dd7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:08.398888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302300c-3a37-40ad-9038-6a2b6264aaa1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:08.412360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:08.431503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:08.445180 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df9523-1968-4cce-9d05-4ec30ff0f16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:08.445948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df9523-1968-4cce-9d05-4ec30ff0f16d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:08.446235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df9523-1968-4cce-9d05-4ec30ff0f16d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:08.446506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df9523-1968-4cce-9d05-4ec30ff0f16d None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:08.448105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df9523-1968-4cce-9d05-4ec30ff0f16d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:08.448105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df9523-1968-4cce-9d05-4ec30ff0f16d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:08.449279 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df9523-1968-4cce-9d05-4ec30ff0f16d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:08.463042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 433/854] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:08.468746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df9523-1968-4cce-9d05-4ec30ff0f16d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:08.530029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579082db-ba9b-4771-9040-85945c662dd7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:08.568193 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94bd67-cf8a-4177-ac85-2bbdc9e3656b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:08.569436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94bd67-cf8a-4177-ac85-2bbdc9e3656b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:08.569947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94bd67-cf8a-4177-ac85-2bbdc9e3656b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:08.570509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94bd67-cf8a-4177-ac85-2bbdc9e3656b None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:08.571483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94bd67-cf8a-4177-ac85-2bbdc9e3656b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:08.572038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94bd67-cf8a-4177-ac85-2bbdc9e3656b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:08.573760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94bd67-cf8a-4177-ac85-2bbdc9e3656b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:08.578443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df9523-1968-4cce-9d05-4ec30ff0f16d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:08.579787 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 422/855] 10.222.0.43 () {56 vars in 1126 bytes} [Mon Jul 4 07:51:08 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:08.632287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342cb5a-322b-42a7-8dae-98c9af483f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:08.633481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342cb5a-322b-42a7-8dae-98c9af483f45 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:08.634058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342cb5a-322b-42a7-8dae-98c9af483f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:08.634581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342cb5a-322b-42a7-8dae-98c9af483f45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:08.635718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342cb5a-322b-42a7-8dae-98c9af483f45 None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=6763884b51204c488ca8c7e452e3a29e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:08.636263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342cb5a-322b-42a7-8dae-98c9af483f45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:08.638036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342cb5a-322b-42a7-8dae-98c9af483f45 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:08.646280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 434/856] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:51:08 2022] GET /identity/v3/role_assignments?user.id=6763884b51204c488ca8c7e452e3a29e&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:09.030149 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212afe24-cc86-4bed-bd70-7e7585a5cac8 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:09.030908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212afe24-cc86-4bed-bd70-7e7585a5cac8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:09.031418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212afe24-cc86-4bed-bd70-7e7585a5cac8 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:09.033411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 423/857] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:09.039248 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9518430-a073-44b0-8dc1-571ff0c0db20 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:09.039823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9518430-a073-44b0-8dc1-571ff0c0db20 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:09.040346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9518430-a073-44b0-8dc1-571ff0c0db20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:09.058964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c9518430-a073-44b0-8dc1-571ff0c0db20 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:09.060615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c9518430-a073-44b0-8dc1-571ff0c0db20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:09.111930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:09.136132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 435/858] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:09.159756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212afe24-cc86-4bed-bd70-7e7585a5cac8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:09.226188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffe9d3-1abc-4750-b4b3-28b9f6a0cff5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:09.226520 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffe9d3-1abc-4750-b4b3-28b9f6a0cff5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:09.226768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffe9d3-1abc-4750-b4b3-28b9f6a0cff5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:09.229187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 436/859] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:09.238992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2338d24-da01-401e-80cb-39d02ecb2bb5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:09.239291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2338d24-da01-401e-80cb-39d02ecb2bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:09.239558 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2338d24-da01-401e-80cb-39d02ecb2bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:09.250239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:09.258866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f2338d24-da01-401e-80cb-39d02ecb2bb5 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:09.260263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f2338d24-da01-401e-80cb-39d02ecb2bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:09.285728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212afe24-cc86-4bed-bd70-7e7585a5cac8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:09.331548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f210e-d1ed-4cf8-b513-97a9acc324b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:09.332890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f210e-d1ed-4cf8-b513-97a9acc324b0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:09.333355 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f210e-d1ed-4cf8-b513-97a9acc324b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:09.333890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f210e-d1ed-4cf8-b513-97a9acc324b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:09.342053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 437/860] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:09.347441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f210e-d1ed-4cf8-b513-97a9acc324b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8a5f210e-d1ed-4cf8-b513-97a9acc324b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:09.355911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:09.356973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:09.357892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:09.357892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:09.357892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:09.357892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:09.357892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:09.357892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:09.357892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:09.357892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:09.357892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:09.357892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:09.357892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:51:09.357892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:09.359490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2338d24-da01-401e-80cb-39d02ecb2bb5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:09.360903 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 424/861] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:51:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:09.366945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f210e-d1ed-4cf8-b513-97a9acc324b0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:09.476940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f210e-d1ed-4cf8-b513-97a9acc324b0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:09.515208 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a2712-0ecc-420d-9a00-a736a5259cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:09.515897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a2712-0ecc-420d-9a00-a736a5259cda None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:09.516176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a2712-0ecc-420d-9a00-a736a5259cda None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:09.516446 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a2712-0ecc-420d-9a00-a736a5259cda None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:09.517205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a2712-0ecc-420d-9a00-a736a5259cda None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:09.517510 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a2712-0ecc-420d-9a00-a736a5259cda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:09.518979 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a2712-0ecc-420d-9a00-a736a5259cda None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:09.520561 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:09.535603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 425/862] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:51:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:09.536694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a2712-0ecc-420d-9a00-a736a5259cda None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:09.627916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2338d24-da01-401e-80cb-39d02ecb2bb5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:09.640576 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:09.665795 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244ed97-4043-4502-90cb-5e67646608c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:09.666558 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244ed97-4043-4502-90cb-5e67646608c4 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:09.666902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244ed97-4043-4502-90cb-5e67646608c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:09.667229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244ed97-4043-4502-90cb-5e67646608c4 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:09.668010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244ed97-4043-4502-90cb-5e67646608c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:09.669509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244ed97-4043-4502-90cb-5e67646608c4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:09.673177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a2712-0ecc-420d-9a00-a736a5259cda None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:09.690125 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 438/863] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:51:09 2022] POST /identity/v3/services => generated 251 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 07:51:09.721361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de2044-6b71-478c-b732-d480af0dde2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:09.722480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de2044-6b71-478c-b732-d480af0dde2e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:09.722966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de2044-6b71-478c-b732-d480af0dde2e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:09.723418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de2044-6b71-478c-b732-d480af0dde2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:09.737055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de2044-6b71-478c-b732-d480af0dde2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:09.738987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de2044-6b71-478c-b732-d480af0dde2e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-97de2044-6b71-478c-b732-d480af0dde2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:09.761116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:09.762370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:09.763606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:09.763606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:09.763606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:09.763606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:09.765801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 426/864] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:09.773686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244ed97-4043-4502-90cb-5e67646608c4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:09.890963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244ed97-4043-4502-90cb-5e67646608c4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:09.926914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f91a08-b574-4567-8de8-cf0abf16f679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:09.927974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f91a08-b574-4567-8de8-cf0abf16f679 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:09.928539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f91a08-b574-4567-8de8-cf0abf16f679 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:09.929167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f91a08-b574-4567-8de8-cf0abf16f679 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:09.930203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f91a08-b574-4567-8de8-cf0abf16f679 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:09.930886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f91a08-b574-4567-8de8-cf0abf16f679 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:09.932571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f91a08-b574-4567-8de8-cf0abf16f679 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:09.947693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 439/865] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:09.953698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de2044-6b71-478c-b732-d480af0dde2e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:10.002853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:10.060869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de2044-6b71-478c-b732-d480af0dde2e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:10.096388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1b46c-bc35-4f20-a420-18f2618a59d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:10.097232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1b46c-bc35-4f20-a420-18f2618a59d9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:10.097517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1b46c-bc35-4f20-a420-18f2618a59d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:10.097815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1b46c-bc35-4f20-a420-18f2618a59d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:10.106175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1b46c-bc35-4f20-a420-18f2618a59d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:10.107650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1b46c-bc35-4f20-a420-18f2618a59d9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-26f1b46c-bc35-4f20-a420-18f2618a59d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:51:10.123397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:10.124445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:10.125431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:10.125431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:10.125431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:51:10.125431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:51:10.125431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:51:10.125431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:51:10.125431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:51:10.125431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:51:10.125431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 07:51:10.125431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:10.125431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 427/866] 10.222.0.43 () {56 vars in 1117 bytes} [Mon Jul 4 07:51:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:10.130901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f91a08-b574-4567-8de8-cf0abf16f679 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:10.170195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:10.244184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f91a08-b574-4567-8de8-cf0abf16f679 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:10.282026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486854ea-afed-41fc-b7f6-2a21ff131da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:10.282761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486854ea-afed-41fc-b7f6-2a21ff131da2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:10.283043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486854ea-afed-41fc-b7f6-2a21ff131da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:10.283316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486854ea-afed-41fc-b7f6-2a21ff131da2 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:10.284168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486854ea-afed-41fc-b7f6-2a21ff131da2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:10.284478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486854ea-afed-41fc-b7f6-2a21ff131da2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:10.285952 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486854ea-afed-41fc-b7f6-2a21ff131da2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:10.306736 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 440/867] 10.222.0.43 () {56 vars in 1149 bytes} [Mon Jul 4 07:51:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:10.312707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1b46c-bc35-4f20-a420-18f2618a59d9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:10.422259 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1b46c-bc35-4f20-a420-18f2618a59d9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:10.424649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:10.466079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883db8f9-8aa3-4cc2-8b38-ce50c56e2452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:10.466752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883db8f9-8aa3-4cc2-8b38-ce50c56e2452 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:10.467036 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883db8f9-8aa3-4cc2-8b38-ce50c56e2452 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:10.467312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883db8f9-8aa3-4cc2-8b38-ce50c56e2452 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:10.545818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883db8f9-8aa3-4cc2-8b38-ce50c56e2452 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:10.547307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883db8f9-8aa3-4cc2-8b38-ce50c56e2452 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:10.658068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 428/868] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:51:10 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:51:10.663421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486854ea-afed-41fc-b7f6-2a21ff131da2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:10.744082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15124378-1294-4c44-8cf1-3c37b4decb96 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:10.744426 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15124378-1294-4c44-8cf1-3c37b4decb96 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:10.744638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15124378-1294-4c44-8cf1-3c37b4decb96 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:10.746393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 429/869] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:10.751713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885937e1-dd99-4b01-9112-7c132fa7c423 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:10.752004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885937e1-dd99-4b01-9112-7c132fa7c423 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:10.752299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885937e1-dd99-4b01-9112-7c132fa7c423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:10.759096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:10.762676 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:10.770737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-885937e1-dd99-4b01-9112-7c132fa7c423 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:10.772006 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-885937e1-dd99-4b01-9112-7c132fa7c423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:10.776800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486854ea-afed-41fc-b7f6-2a21ff131da2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:10.816681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5fd37-36d7-4b9b-9da3-46f983202fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:10.817538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5fd37-36d7-4b9b-9da3-46f983202fec None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:10.817853 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5fd37-36d7-4b9b-9da3-46f983202fec None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:10.818129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5fd37-36d7-4b9b-9da3-46f983202fec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:10.830312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5fd37-36d7-4b9b-9da3-46f983202fec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:10.831992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5fd37-36d7-4b9b-9da3-46f983202fec None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:10.846470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 430/870] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:10.847607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d888f6d-9f82-438f-b640-a35ea71016fe None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:10.847985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d888f6d-9f82-438f-b640-a35ea71016fe None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:10.848182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d888f6d-9f82-438f-b640-a35ea71016fe None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:10.849750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 431/871] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-33f5fd37-36d7-4b9b-9da3-46f983202fec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:10.852005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:10.853043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:10.853983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:10.853983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:10.853983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:10.853983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:10.853983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 441/872] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:10.858872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d888f6d-9f82-438f-b640-a35ea71016fe None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:10.859418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88812ce4-36fe-4c9e-8441-8d4bb6ad2e6c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:10.859710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88812ce4-36fe-4c9e-8441-8d4bb6ad2e6c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:10.859987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88812ce4-36fe-4c9e-8441-8d4bb6ad2e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:10.881772 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-88812ce4-36fe-4c9e-8441-8d4bb6ad2e6c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:10.883318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-88812ce4-36fe-4c9e-8441-8d4bb6ad2e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:10.979103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 442/873] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:10.989961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88812ce4-36fe-4c9e-8441-8d4bb6ad2e6c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:11.002263 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d888f6d-9f82-438f-b640-a35ea71016fe None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:11.050108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fb9b5-9007-43ea-90ee-c10466c13e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:11.050794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fb9b5-9007-43ea-90ee-c10466c13e99 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:11.051078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fb9b5-9007-43ea-90ee-c10466c13e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:11.051356 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fb9b5-9007-43ea-90ee-c10466c13e99 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:11.052085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4fb9b5-9007-43ea-90ee-c10466c13e99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:11.052395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4fb9b5-9007-43ea-90ee-c10466c13e99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:11.053920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4fb9b5-9007-43ea-90ee-c10466c13e99 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:11.055507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:11.071166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 432/874] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:11.071166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fb9b5-9007-43ea-90ee-c10466c13e99 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:11.080820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:11.121188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88812ce4-36fe-4c9e-8441-8d4bb6ad2e6c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:11.162353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f1009-c5bf-443f-9bb6-8835c247253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:11.163542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f1009-c5bf-443f-9bb6-8835c247253a None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:11.163820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f1009-c5bf-443f-9bb6-8835c247253a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:11.164099 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f1009-c5bf-443f-9bb6-8835c247253a None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:11.164891 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f1009-c5bf-443f-9bb6-8835c247253a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:11.166411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f1009-c5bf-443f-9bb6-8835c247253a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:11.184829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 443/875] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:51:10 2022] POST /identity/v3/services => generated 269 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:11.185892 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f1009-c5bf-443f-9bb6-8835c247253a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:11.198394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fb9b5-9007-43ea-90ee-c10466c13e99 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:11.253194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8afc98-e22f-4f4d-8e9e-b56abc3b90eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:11.255057 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8afc98-e22f-4f4d-8e9e-b56abc3b90eb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:11.255407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8afc98-e22f-4f4d-8e9e-b56abc3b90eb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:11.255758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8afc98-e22f-4f4d-8e9e-b56abc3b90eb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:11.256694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8afc98-e22f-4f4d-8e9e-b56abc3b90eb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:11.258421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8afc98-e22f-4f4d-8e9e-b56abc3b90eb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8a8afc98-e22f-4f4d-8e9e-b56abc3b90eb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:11.272758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:11.274202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:11.275197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:11.278335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 433/876] 10.222.0.43 () {56 vars in 1126 bytes} [Mon Jul 4 07:51:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:11.281265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8afc98-e22f-4f4d-8e9e-b56abc3b90eb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:11.348662 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f1009-c5bf-443f-9bb6-8835c247253a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:11.353368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:11.386302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:11.394796 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5810b-8e82-4168-9c3e-7a3b386310e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:11.396166 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5810b-8e82-4168-9c3e-7a3b386310e0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:11.396636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5810b-8e82-4168-9c3e-7a3b386310e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:11.397145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5810b-8e82-4168-9c3e-7a3b386310e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:11.406562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8afc98-e22f-4f4d-8e9e-b56abc3b90eb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:11.408280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5810b-8e82-4168-9c3e-7a3b386310e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:11.410196 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5810b-8e82-4168-9c3e-7a3b386310e0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-24a5810b-8e82-4168-9c3e-7a3b386310e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:11.426668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:11.427823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:11.429017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:11.429017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:11.429017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:11.429017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:11.429017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:11.429017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:11.429017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:11.429017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:11.433215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 444/877] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:11.438459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5810b-8e82-4168-9c3e-7a3b386310e0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:11.451986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdac790-793c-4f89-9500-fef5f187d1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:11.452671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdac790-793c-4f89-9500-fef5f187d1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:11.453220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdac790-793c-4f89-9500-fef5f187d1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:11.453414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdac790-793c-4f89-9500-fef5f187d1a3 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:11.454232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdac790-793c-4f89-9500-fef5f187d1a3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:11.454542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdac790-793c-4f89-9500-fef5f187d1a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:11.456056 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdac790-793c-4f89-9500-fef5f187d1a3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:11.463191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 434/878] 10.222.0.43 () {56 vars in 1122 bytes} [Mon Jul 4 07:51:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:11.469166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdac790-793c-4f89-9500-fef5f187d1a3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:11.563877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5810b-8e82-4168-9c3e-7a3b386310e0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:11.614805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdac790-793c-4f89-9500-fef5f187d1a3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:11.629007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6f979-4d9f-48c1-8abc-0a9a1ae5d238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:11.630322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6f979-4d9f-48c1-8abc-0a9a1ae5d238 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:11.631107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6f979-4d9f-48c1-8abc-0a9a1ae5d238 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:11.631444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6f979-4d9f-48c1-8abc-0a9a1ae5d238 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:11.633362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6f979-4d9f-48c1-8abc-0a9a1ae5d238 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:11.633705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6f979-4d9f-48c1-8abc-0a9a1ae5d238 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:11.635682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:11.636573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6f979-4d9f-48c1-8abc-0a9a1ae5d238 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:11.659193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 445/879] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:51:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:11.660314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6f979-4d9f-48c1-8abc-0a9a1ae5d238 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:11.660428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e354b7-5fd0-48da-9e0c-cbae03a9fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:11.661031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e354b7-5fd0-48da-9e0c-cbae03a9fcea None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:11.661319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e354b7-5fd0-48da-9e0c-cbae03a9fcea None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:11.661912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e354b7-5fd0-48da-9e0c-cbae03a9fcea None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:11.662717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e354b7-5fd0-48da-9e0c-cbae03a9fcea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:11.664082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e354b7-5fd0-48da-9e0c-cbae03a9fcea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:11.666507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e354b7-5fd0-48da-9e0c-cbae03a9fcea None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:11.675142 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 435/880] 10.222.0.43 () {56 vars in 1122 bytes} [Mon Jul 4 07:51:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:11.727627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:11.792542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6f979-4d9f-48c1-8abc-0a9a1ae5d238 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:11.832317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ad3d8-e924-45a9-ad12-530f7f7ce274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:11.832950 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ad3d8-e924-45a9-ad12-530f7f7ce274 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:11.833229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ad3d8-e924-45a9-ad12-530f7f7ce274 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:11.833504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ad3d8-e924-45a9-ad12-530f7f7ce274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:11.834391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ad3d8-e924-45a9-ad12-530f7f7ce274 None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=80a45a063c0e4f1e9b2156fb2f4cd8b8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:11.834798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ad3d8-e924-45a9-ad12-530f7f7ce274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:11.836447 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ad3d8-e924-45a9-ad12-530f7f7ce274 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:11.844798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 446/881] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:51:11 2022] GET /identity/v3/role_assignments?user.id=80a45a063c0e4f1e9b2156fb2f4cd8b8&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:11.844999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2a4d2-1c26-44a7-9480-67031fdfba4d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:11.845161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2a4d2-1c26-44a7-9480-67031fdfba4d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:11.845444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2a4d2-1c26-44a7-9480-67031fdfba4d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:11.847025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 436/882] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:11.852639 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55d957-483b-4856-ace5-fbeb28160a77 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:11.852992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55d957-483b-4856-ace5-fbeb28160a77 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:11.853209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55d957-483b-4856-ace5-fbeb28160a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:11.872145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6c55d957-483b-4856-ace5-fbeb28160a77 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:11.873383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6c55d957-483b-4856-ace5-fbeb28160a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:11.955182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 447/883] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:11.979464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2a4d2-1c26-44a7-9480-67031fdfba4d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:12.094709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2a4d2-1c26-44a7-9480-67031fdfba4d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:12.139730 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa80fd-eb7e-478b-a91a-9ae0d6607bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:12.140556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa80fd-eb7e-478b-a91a-9ae0d6607bbc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:12.140841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa80fd-eb7e-478b-a91a-9ae0d6607bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:12.141167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa80fd-eb7e-478b-a91a-9ae0d6607bbc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:12.141896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa80fd-eb7e-478b-a91a-9ae0d6607bbc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:12.143317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa80fd-eb7e-478b-a91a-9ae0d6607bbc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-aeaa80fd-eb7e-478b-a91a-9ae0d6607bbc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:12.155746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:12.156799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:12.158104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 437/884] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:51:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:12.163516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55d957-483b-4856-ace5-fbeb28160a77 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:12.214120 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:12.282759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55d957-483b-4856-ace5-fbeb28160a77 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:12.322188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdb9d3-4bb4-42d0-adad-aa1559d2a8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:12.322869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdb9d3-4bb4-42d0-adad-aa1559d2a8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:12.323146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdb9d3-4bb4-42d0-adad-aa1559d2a8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:12.323435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdb9d3-4bb4-42d0-adad-aa1559d2a8a0 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:12.324159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdb9d3-4bb4-42d0-adad-aa1559d2a8a0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:12.324459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdb9d3-4bb4-42d0-adad-aa1559d2a8a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:12.325868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdb9d3-4bb4-42d0-adad-aa1559d2a8a0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:12.331372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 448/885] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:51:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:12.337079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa80fd-eb7e-478b-a91a-9ae0d6607bbc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:12.386709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:12.444435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa80fd-eb7e-478b-a91a-9ae0d6607bbc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:12.482707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e172759-e86a-415e-bf37-50d0125c3d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:12.483396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172759-e86a-415e-bf37-50d0125c3d20 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:12.483679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172759-e86a-415e-bf37-50d0125c3d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:12.483953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172759-e86a-415e-bf37-50d0125c3d20 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:12.484672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e172759-e86a-415e-bf37-50d0125c3d20 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:12.484980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e172759-e86a-415e-bf37-50d0125c3d20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:12.486397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e172759-e86a-415e-bf37-50d0125c3d20 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:12.492485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 438/886] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:51:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:12.498029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdb9d3-4bb4-42d0-adad-aa1559d2a8a0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:12.601425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdb9d3-4bb4-42d0-adad-aa1559d2a8a0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:12.611102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:12.637712 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accef54-5173-4657-ba5a-fb4dd29ea265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:12.638305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accef54-5173-4657-ba5a-fb4dd29ea265 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:12.638585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accef54-5173-4657-ba5a-fb4dd29ea265 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:12.638855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accef54-5173-4657-ba5a-fb4dd29ea265 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:12.639640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accef54-5173-4657-ba5a-fb4dd29ea265 None admin] RBAC: Adding query filter params (interface=public, service_id=7ff33c00f7a84b74a339bd74e663bf0a, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:12.639951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accef54-5173-4657-ba5a-fb4dd29ea265 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:12.641321 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accef54-5173-4657-ba5a-fb4dd29ea265 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:12.647358 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 449/887] 10.222.0.43 () {56 vars in 1251 bytes} [Mon Jul 4 07:51:12 2022] GET /identity/v3/endpoints?service_id=7ff33c00f7a84b74a339bd74e663bf0a&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:12.652749 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e172759-e86a-415e-bf37-50d0125c3d20 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:12.753174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e172759-e86a-415e-bf37-50d0125c3d20 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:12.764783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:12.789346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb7b20-b90f-4a6b-957e-6b54b5ee3bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:12.790055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb7b20-b90f-4a6b-957e-6b54b5ee3bfa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:12.790330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb7b20-b90f-4a6b-957e-6b54b5ee3bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:12.790610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb7b20-b90f-4a6b-957e-6b54b5ee3bfa None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:12.791305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb7b20-b90f-4a6b-957e-6b54b5ee3bfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:12.792693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb7b20-b90f-4a6b-957e-6b54b5ee3bfa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:12.799241 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 439/888] 10.222.0.43 () {56 vars in 1087 bytes} [Mon Jul 4 07:51:12 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 07:51:13.428308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24609ea4-8275-4af6-8a34-6d4b7ebf3dd8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:13.428821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24609ea4-8275-4af6-8a34-6d4b7ebf3dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:13.429088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24609ea4-8275-4af6-8a34-6d4b7ebf3dd8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:13.430648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 450/889] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:13.436740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c9e30-b478-48b5-b3f5-d0b53ba3afc4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:13.437225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c9e30-b478-48b5-b3f5-d0b53ba3afc4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:13.437422 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c9e30-b478-48b5-b3f5-d0b53ba3afc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:13.456612 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9e7c9e30-b478-48b5-b3f5-d0b53ba3afc4 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:13.457842 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9e7c9e30-b478-48b5-b3f5-d0b53ba3afc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:13.528740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 440/890] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:13.551604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24609ea4-8275-4af6-8a34-6d4b7ebf3dd8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:13.671592 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24609ea4-8275-4af6-8a34-6d4b7ebf3dd8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:13.713487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d5ce5-c604-4829-aea0-0f2434735b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:13.714326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d5ce5-c604-4829-aea0-0f2434735b39 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:13.714737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d5ce5-c604-4829-aea0-0f2434735b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:13.715010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d5ce5-c604-4829-aea0-0f2434735b39 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:13.715671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d5ce5-c604-4829-aea0-0f2434735b39 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:13.717112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d5ce5-c604-4829-aea0-0f2434735b39 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f10d5ce5-c604-4829-aea0-0f2434735b39 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:13.730985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:13.731997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:13.732876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 451/891] 10.222.0.43 () {56 vars in 1132 bytes} [Mon Jul 4 07:51:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:13.740470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c9e30-b478-48b5-b3f5-d0b53ba3afc4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:13.848576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c9e30-b478-48b5-b3f5-d0b53ba3afc4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:13.851423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:13.885688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d1e33-c590-4087-887a-faefdfa1ab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:13.886389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d1e33-c590-4087-887a-faefdfa1ab66 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:13.886671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d1e33-c590-4087-887a-faefdfa1ab66 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:13.886944 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d1e33-c590-4087-887a-faefdfa1ab66 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:13.887663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d1e33-c590-4087-887a-faefdfa1ab66 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:13.887969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d1e33-c590-4087-887a-faefdfa1ab66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:13.889366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d1e33-c590-4087-887a-faefdfa1ab66 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:13.895062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 441/892] 10.222.0.43 () {56 vars in 1126 bytes} [Mon Jul 4 07:51:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:13.901094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d5ce5-c604-4829-aea0-0f2434735b39 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:13.901797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684c8d0-4a97-4df2-b5ca-ef3b93e061b7 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:13.902164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684c8d0-4a97-4df2-b5ca-ef3b93e061b7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:13.902444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684c8d0-4a97-4df2-b5ca-ef3b93e061b7 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:13.905401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 442/893] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:13.910279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc76588-6405-46c4-9b91-5b26e38af99b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:13.910565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc76588-6405-46c4-9b91-5b26e38af99b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:13.910846 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc76588-6405-46c4-9b91-5b26e38af99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:13.930078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9dc76588-6405-46c4-9b91-5b26e38af99b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:13.931454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9dc76588-6405-46c4-9b91-5b26e38af99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:13.946191 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:14.009759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 443/894] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:14.017467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d5ce5-c604-4829-aea0-0f2434735b39 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:14.029015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc76588-6405-46c4-9b91-5b26e38af99b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:14.055766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95564624-891a-4873-bab5-7e82bd756cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:14.056467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95564624-891a-4873-bab5-7e82bd756cdd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:14.056813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95564624-891a-4873-bab5-7e82bd756cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:14.057088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95564624-891a-4873-bab5-7e82bd756cdd None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:14.057952 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95564624-891a-4873-bab5-7e82bd756cdd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:14.058263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95564624-891a-4873-bab5-7e82bd756cdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:14.059743 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95564624-891a-4873-bab5-7e82bd756cdd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:14.065880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 452/895] 10.222.0.43 () {56 vars in 1126 bytes} [Mon Jul 4 07:51:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:14.072013 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95564624-891a-4873-bab5-7e82bd756cdd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:14.151465 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:14.155785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc76588-6405-46c4-9b91-5b26e38af99b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:14.177670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:14.181024 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95564624-891a-4873-bab5-7e82bd756cdd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:14.196766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96eaf06-6a9d-45ae-89de-b893cdc1f575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:14.197299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eaf06-6a9d-45ae-89de-b893cdc1f575 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:14.197582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eaf06-6a9d-45ae-89de-b893cdc1f575 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:14.197888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eaf06-6a9d-45ae-89de-b893cdc1f575 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:14.198630 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96eaf06-6a9d-45ae-89de-b893cdc1f575 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:14.200010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96eaf06-6a9d-45ae-89de-b893cdc1f575 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:14.225730 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 444/896] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:51:14 2022] POST /identity/v3/services => generated 241 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:14.232593 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa95803-b2e4-486b-a440-fde536caf139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:14.233737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa95803-b2e4-486b-a440-fde536caf139 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:14.234264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa95803-b2e4-486b-a440-fde536caf139 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:14.234789 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa95803-b2e4-486b-a440-fde536caf139 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:14.235820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa95803-b2e4-486b-a440-fde536caf139 None admin] RBAC: Adding query filter params (interface=public, service_id=cd30191b56274f13b6b5e8d5ffaa7020, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:14.236393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa95803-b2e4-486b-a440-fde536caf139 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:14.238176 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa95803-b2e4-486b-a440-fde536caf139 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:14.245800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 453/897] 10.222.0.43 () {56 vars in 1251 bytes} [Mon Jul 4 07:51:14 2022] GET /identity/v3/endpoints?service_id=cd30191b56274f13b6b5e8d5ffaa7020&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:14.250704 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aca971-c707-4081-8d05-7470dfbb54c7 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:14.251268 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aca971-c707-4081-8d05-7470dfbb54c7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:14.251508 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa95803-b2e4-486b-a440-fde536caf139 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:14.252078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aca971-c707-4081-8d05-7470dfbb54c7 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:14.254621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 445/898] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:14.260837 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f2769-d21b-45e4-80cf-13057a3dd504 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:14.261470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f2769-d21b-45e4-80cf-13057a3dd504 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:14.261992 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f2769-d21b-45e4-80cf-13057a3dd504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:14.282267 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e39f2769-d21b-45e4-80cf-13057a3dd504 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:14.284027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e39f2769-d21b-45e4-80cf-13057a3dd504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:14.361398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 446/899] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:14.383008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa95803-b2e4-486b-a440-fde536caf139 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:14.386607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f2769-d21b-45e4-80cf-13057a3dd504 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:14.435084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c5568-ea70-4d47-827e-0a716c268bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:14.436065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c5568-ea70-4d47-827e-0a716c268bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:14.436356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c5568-ea70-4d47-827e-0a716c268bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:14.436632 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c5568-ea70-4d47-827e-0a716c268bc8 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:14.437603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c5568-ea70-4d47-827e-0a716c268bc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:14.439314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c5568-ea70-4d47-827e-0a716c268bc8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:14.447177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 454/900] 10.222.0.43 () {56 vars in 1087 bytes} [Mon Jul 4 07:51:14 2022] GET /identity/v3/services => generated 1497 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 07:51:14.492442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:14.514703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f2769-d21b-45e4-80cf-13057a3dd504 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:14.555256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11386648-5825-4403-a610-774ca2481958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:14.556109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11386648-5825-4403-a610-774ca2481958 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:14.556389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11386648-5825-4403-a610-774ca2481958 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:14.556664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11386648-5825-4403-a610-774ca2481958 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:14.569182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11386648-5825-4403-a610-774ca2481958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-11386648-5825-4403-a610-774ca2481958 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:14.577718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:14.578741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:14.579698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 447/901] 10.222.0.43 () {56 vars in 1120 bytes} [Mon Jul 4 07:51:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:14.585646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c5568-ea70-4d47-827e-0a716c268bc8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:14.622056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:14.699599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c5568-ea70-4d47-827e-0a716c268bc8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:14.736626 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0a887-30a0-4147-98c0-0611652183a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:14.737760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0a887-30a0-4147-98c0-0611652183a5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:14.738268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0a887-30a0-4147-98c0-0611652183a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:14.738756 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0a887-30a0-4147-98c0-0611652183a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:14.739794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0a887-30a0-4147-98c0-0611652183a5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:14.740400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0a887-30a0-4147-98c0-0611652183a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:14.742281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0a887-30a0-4147-98c0-0611652183a5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:14.752302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 455/902] 10.222.0.43 () {56 vars in 1115 bytes} [Mon Jul 4 07:51:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:14.758819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11386648-5825-4403-a610-774ca2481958 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:14.878482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11386648-5825-4403-a610-774ca2481958 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:14.887548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816e0fc-83b9-421e-9721-b9313fe22c98 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:14.887861 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816e0fc-83b9-421e-9721-b9313fe22c98 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:14.888104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816e0fc-83b9-421e-9721-b9313fe22c98 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:14.889748 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 456/903] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:14.894946 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc873d0-2d4a-4b31-a392-54edce81f887 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:14.895229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc873d0-2d4a-4b31-a392-54edce81f887 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:14.895505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc873d0-2d4a-4b31-a392-54edce81f887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:14.913181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bbc873d0-2d4a-4b31-a392-54edce81f887 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:14.914583 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bbc873d0-2d4a-4b31-a392-54edce81f887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:14.918871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6483dfe-6560-4d59-8914-aa8d54853af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:14.919939 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6483dfe-6560-4d59-8914-aa8d54853af6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:14.920451 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6483dfe-6560-4d59-8914-aa8d54853af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:14.920927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6483dfe-6560-4d59-8914-aa8d54853af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:14.932459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6483dfe-6560-4d59-8914-aa8d54853af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:14.934305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6483dfe-6560-4d59-8914-aa8d54853af6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:14.942816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-d6483dfe-6560-4d59-8914-aa8d54853af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:14.952870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:14.954119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:14.955277 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:14.955277 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:14.955277 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:14.955277 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:14.957445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 448/904] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:14.964174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6483dfe-6560-4d59-8914-aa8d54853af6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:15.001646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 457/905] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:15.018331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc873d0-2d4a-4b31-a392-54edce81f887 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:15.051205 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:15.098738 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6483dfe-6560-4d59-8914-aa8d54853af6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:15.144287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc3ff8-caa2-47a7-ba8c-a09a595cd489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:15.145375 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc3ff8-caa2-47a7-ba8c-a09a595cd489 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:15.145888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc3ff8-caa2-47a7-ba8c-a09a595cd489 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:15.146454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc3ff8-caa2-47a7-ba8c-a09a595cd489 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:15.147562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc3ff8-caa2-47a7-ba8c-a09a595cd489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:15.148067 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc3ff8-caa2-47a7-ba8c-a09a595cd489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:15.149791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc3ff8-caa2-47a7-ba8c-a09a595cd489 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:15.153538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc873d0-2d4a-4b31-a392-54edce81f887 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:15.168409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 449/906] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:15.169799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc3ff8-caa2-47a7-ba8c-a09a595cd489 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:15.191640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e655c-f781-47e4-95c1-ac4b7943573f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:15.192491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e655c-f781-47e4-95c1-ac4b7943573f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:15.192783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e655c-f781-47e4-95c1-ac4b7943573f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:15.193055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e655c-f781-47e4-95c1-ac4b7943573f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:15.193753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e655c-f781-47e4-95c1-ac4b7943573f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:15.195730 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e655c-f781-47e4-95c1-ac4b7943573f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f03e655c-f781-47e4-95c1-ac4b7943573f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:15.209211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:15.211119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 07:51:15.212027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:15.213841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 458/907] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:51:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:15.217235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e655c-f781-47e4-95c1-ac4b7943573f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:15.240306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:15.285441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc3ff8-caa2-47a7-ba8c-a09a595cd489 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:15.309557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:15.325289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ffe68-99f3-49ba-9de1-e3c05141a582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:15.326430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ffe68-99f3-49ba-9de1-e3c05141a582 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:15.326815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ffe68-99f3-49ba-9de1-e3c05141a582 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:15.327193 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ffe68-99f3-49ba-9de1-e3c05141a582 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:15.332882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e655c-f781-47e4-95c1-ac4b7943573f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:15.336262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ffe68-99f3-49ba-9de1-e3c05141a582 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:15.337551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ffe68-99f3-49ba-9de1-e3c05141a582 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-271ffe68-99f3-49ba-9de1-e3c05141a582 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:51:15.354308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:15.355544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:15.356724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:15.356724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:15.356724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:51:15.356724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:51:15.356724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:51:15.356724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:51:15.356724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:51:15.356724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:51:15.356724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 07:51:15.356724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:15.362153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 450/908] 10.222.0.43 () {56 vars in 1117 bytes} [Mon Jul 4 07:51:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:15.364578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ffe68-99f3-49ba-9de1-e3c05141a582 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:15.373450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b0a45-b95d-4d9e-a535-cb0ac5595694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:15.374517 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b0a45-b95d-4d9e-a535-cb0ac5595694 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:15.374966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b0a45-b95d-4d9e-a535-cb0ac5595694 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:15.375379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b0a45-b95d-4d9e-a535-cb0ac5595694 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:15.376253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b0a45-b95d-4d9e-a535-cb0ac5595694 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:15.376738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b0a45-b95d-4d9e-a535-cb0ac5595694 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:15.378407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b0a45-b95d-4d9e-a535-cb0ac5595694 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:15.389161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 459/909] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:51:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:15.390408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b0a45-b95d-4d9e-a535-cb0ac5595694 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:15.475110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ffe68-99f3-49ba-9de1-e3c05141a582 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:15.499711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:15.508903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b0a45-b95d-4d9e-a535-cb0ac5595694 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:15.516242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45397d43-dfc9-4974-8934-5ff6e1cac0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:15.517043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45397d43-dfc9-4974-8934-5ff6e1cac0ba None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:15.517428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45397d43-dfc9-4974-8934-5ff6e1cac0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:15.517832 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45397d43-dfc9-4974-8934-5ff6e1cac0ba None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:15.518897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45397d43-dfc9-4974-8934-5ff6e1cac0ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:15.519307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45397d43-dfc9-4974-8934-5ff6e1cac0ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:15.520882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45397d43-dfc9-4974-8934-5ff6e1cac0ba None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:15.545142 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 451/910] 10.222.0.43 () {56 vars in 1149 bytes} [Mon Jul 4 07:51:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:15.546250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45397d43-dfc9-4974-8934-5ff6e1cac0ba None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:15.548643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc7529-9425-4df0-b3af-28ca3c26064a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:15.549683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc7529-9425-4df0-b3af-28ca3c26064a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:15.550343 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc7529-9425-4df0-b3af-28ca3c26064a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:15.550767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc7529-9425-4df0-b3af-28ca3c26064a None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:15.551606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc7529-9425-4df0-b3af-28ca3c26064a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:15.552094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc7529-9425-4df0-b3af-28ca3c26064a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:15.553813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc7529-9425-4df0-b3af-28ca3c26064a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:15.564587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 460/911] 10.222.0.43 () {56 vars in 1114 bytes} [Mon Jul 4 07:51:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:15.565694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc7529-9425-4df0-b3af-28ca3c26064a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:15.578246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:15.657790 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45397d43-dfc9-4974-8934-5ff6e1cac0ba None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:15.678392 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc7529-9425-4df0-b3af-28ca3c26064a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:15.697374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15219741-4184-4e75-a3b0-58075415370a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:15.698314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15219741-4184-4e75-a3b0-58075415370a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:15.698694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15219741-4184-4e75-a3b0-58075415370a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:15.699086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15219741-4184-4e75-a3b0-58075415370a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:15.717536 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5eddda-9fd3-47aa-a1b7-afcd49f9ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:15.718613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5eddda-9fd3-47aa-a1b7-afcd49f9ade2 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:15.719044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5eddda-9fd3-47aa-a1b7-afcd49f9ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:15.719453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5eddda-9fd3-47aa-a1b7-afcd49f9ade2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:15.720616 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5eddda-9fd3-47aa-a1b7-afcd49f9ade2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:15.722336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5eddda-9fd3-47aa-a1b7-afcd49f9ade2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:15.757752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 461/912] 10.222.0.43 () {60 vars in 1144 bytes} [Mon Jul 4 07:51:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:15.760455 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:15.784100 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15219741-4184-4e75-a3b0-58075415370a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:15.785825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15219741-4184-4e75-a3b0-58075415370a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:15.901499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 452/913] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:51:15 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:51:15.907551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5eddda-9fd3-47aa-a1b7-afcd49f9ade2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:15.980336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:16.058225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5eddda-9fd3-47aa-a1b7-afcd49f9ade2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:16.102717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27985319-7cb0-4b3d-b85a-a4557c5ff60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:16.103830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985319-7cb0-4b3d-b85a-a4557c5ff60a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:16.104284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985319-7cb0-4b3d-b85a-a4557c5ff60a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:16.104715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985319-7cb0-4b3d-b85a-a4557c5ff60a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:16.117884 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27985319-7cb0-4b3d-b85a-a4557c5ff60a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:16.119405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27985319-7cb0-4b3d-b85a-a4557c5ff60a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-27985319-7cb0-4b3d-b85a-a4557c5ff60a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:16.139202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:16.140213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:16.141149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:16.141149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:16.141149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:16.141149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:16.141149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 462/914] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:16.147710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15219741-4184-4e75-a3b0-58075415370a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:16.238229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:16.263688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15219741-4184-4e75-a3b0-58075415370a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:16.301439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c6803-007f-4071-b871-0649664de4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:16.302194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c6803-007f-4071-b871-0649664de4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:16.302480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c6803-007f-4071-b871-0649664de4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:16.302756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c6803-007f-4071-b871-0649664de4b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:16.303570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c6803-007f-4071-b871-0649664de4b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:16.303937 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c6803-007f-4071-b871-0649664de4b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:16.305555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c6803-007f-4071-b871-0649664de4b6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:16.319338 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 453/915] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:16.325186 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27985319-7cb0-4b3d-b85a-a4557c5ff60a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:16.441341 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b4efa-855d-4129-ad3d-71b1756fc7ce None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:16.441719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b4efa-855d-4129-ad3d-71b1756fc7ce None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:16.442036 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b4efa-855d-4129-ad3d-71b1756fc7ce None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:16.444383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 454/916] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:16.451761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48249af-fc9b-4d9b-90fc-4eb12a88de7f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:16.452122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48249af-fc9b-4d9b-90fc-4eb12a88de7f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:16.452345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48249af-fc9b-4d9b-90fc-4eb12a88de7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:16.454347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27985319-7cb0-4b3d-b85a-a4557c5ff60a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:16.469878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:16.472022 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f48249af-fc9b-4d9b-90fc-4eb12a88de7f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:16.475250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f48249af-fc9b-4d9b-90fc-4eb12a88de7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:16.495238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcf6bc-c056-4723-8179-f45b73d917aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:16.496155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcf6bc-c056-4723-8179-f45b73d917aa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:16.496501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcf6bc-c056-4723-8179-f45b73d917aa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:16.496839 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcf6bc-c056-4723-8179-f45b73d917aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:16.507401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcf6bc-c056-4723-8179-f45b73d917aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:16.509309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcf6bc-c056-4723-8179-f45b73d917aa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-dbfcf6bc-c056-4723-8179-f45b73d917aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:16.527691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:16.528744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:16.529751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:16.529751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:16.529751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:16.529751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:16.529751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:16.529751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:16.529751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:16.529751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:16.533759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 463/917] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:16.536914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcf6bc-c056-4723-8179-f45b73d917aa None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:16.590111 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 455/918] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:16.607386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48249af-fc9b-4d9b-90fc-4eb12a88de7f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:16.655246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcf6bc-c056-4723-8179-f45b73d917aa None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:16.685477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:16.693578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d32cb9-946e-42fd-951f-63aabdd8e82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:16.694506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d32cb9-946e-42fd-951f-63aabdd8e82f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:16.695072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d32cb9-946e-42fd-951f-63aabdd8e82f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:16.695761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d32cb9-946e-42fd-951f-63aabdd8e82f None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:16.696155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d32cb9-946e-42fd-951f-63aabdd8e82f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:16.696472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d32cb9-946e-42fd-951f-63aabdd8e82f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:16.697966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d32cb9-946e-42fd-951f-63aabdd8e82f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:16.716910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 464/919] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:51:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:16.716910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388643c-85b1-4711-9442-c5b21dd70cdb None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:16.716910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388643c-85b1-4711-9442-c5b21dd70cdb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:16.716910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388643c-85b1-4711-9442-c5b21dd70cdb None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:16.716910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 465/920] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:16.718644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388643c-85b1-4711-9442-c5b21dd70cdb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:16.732813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48249af-fc9b-4d9b-90fc-4eb12a88de7f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:16.753644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:16.777853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736a08b-f82a-4b90-8153-107a1d726571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:16.778663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736a08b-f82a-4b90-8153-107a1d726571 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:16.778949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736a08b-f82a-4b90-8153-107a1d726571 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:16.779224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736a08b-f82a-4b90-8153-107a1d726571 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:16.779881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736a08b-f82a-4b90-8153-107a1d726571 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:16.781280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736a08b-f82a-4b90-8153-107a1d726571 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-4736a08b-f82a-4b90-8153-107a1d726571 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:16.794694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:16.795803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:16.796696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 456/921] 10.222.0.43 () {56 vars in 1126 bytes} [Mon Jul 4 07:51:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:16.797527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7384b9f-e51a-40a2-b5ac-6d67b2a5828b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:16.797527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7384b9f-e51a-40a2-b5ac-6d67b2a5828b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:16.798272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7384b9f-e51a-40a2-b5ac-6d67b2a5828b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:16.818877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f7384b9f-e51a-40a2-b5ac-6d67b2a5828b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:16.820378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f7384b9f-e51a-40a2-b5ac-6d67b2a5828b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:16.858380 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388643c-85b1-4711-9442-c5b21dd70cdb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:16.899120 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bb764-c171-4bbc-ae43-7d6ef96fb8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:16.899797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bb764-c171-4bbc-ae43-7d6ef96fb8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:16.900076 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bb764-c171-4bbc-ae43-7d6ef96fb8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:16.900347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bb764-c171-4bbc-ae43-7d6ef96fb8d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:16.901202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bb764-c171-4bbc-ae43-7d6ef96fb8d4 None admin] RBAC: Adding query filter params (role.id=41ba7c35817d4f368d33695911c9295e, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=80a45a063c0e4f1e9b2156fb2f4cd8b8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:16.901514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bb764-c171-4bbc-ae43-7d6ef96fb8d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:16.903062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bb764-c171-4bbc-ae43-7d6ef96fb8d4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:16.905031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 457/922] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:16.907492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7384b9f-e51a-40a2-b5ac-6d67b2a5828b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:16.917662 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 466/923] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:51:16 2022] GET /identity/v3/role_assignments?user.id=80a45a063c0e4f1e9b2156fb2f4cd8b8&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=41ba7c35817d4f368d33695911c9295e => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:16.931234 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bb764-c171-4bbc-ae43-7d6ef96fb8d4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:16.990260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:17.045162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7384b9f-e51a-40a2-b5ac-6d67b2a5828b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:17.058386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bb764-c171-4bbc-ae43-7d6ef96fb8d4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:17.062841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:17.098920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f46386-2241-4b51-9eed-cb2106dfcda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:17.100077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f46386-2241-4b51-9eed-cb2106dfcda2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:17.100588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f46386-2241-4b51-9eed-cb2106dfcda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:17.101080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f46386-2241-4b51-9eed-cb2106dfcda2 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:17.102242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f46386-2241-4b51-9eed-cb2106dfcda2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:17.102407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d9ce9-56d7-45b3-9b8f-04ac5e5e48e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:17.103055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f46386-2241-4b51-9eed-cb2106dfcda2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:17.103555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d9ce9-56d7-45b3-9b8f-04ac5e5e48e3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:17.103909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d9ce9-56d7-45b3-9b8f-04ac5e5e48e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:17.104258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d9ce9-56d7-45b3-9b8f-04ac5e5e48e3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:17.105034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d9ce9-56d7-45b3-9b8f-04ac5e5e48e3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:17.105577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f46386-2241-4b51-9eed-cb2106dfcda2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:17.109387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d9ce9-56d7-45b3-9b8f-04ac5e5e48e3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e49d9ce9-56d7-45b3-9b8f-04ac5e5e48e3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:17.123867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:17.124956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 07:51:17.125883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:17.133106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 467/924] 10.222.0.43 () {56 vars in 1132 bytes} [Mon Jul 4 07:51:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:17.133106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d9ce9-56d7-45b3-9b8f-04ac5e5e48e3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:17.133821 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 458/925] 10.222.0.43 () {56 vars in 1122 bytes} [Mon Jul 4 07:51:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:17.135975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f46386-2241-4b51-9eed-cb2106dfcda2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:17.264127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d9ce9-56d7-45b3-9b8f-04ac5e5e48e3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:17.287192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f46386-2241-4b51-9eed-cb2106dfcda2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:17.317886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d46761-ef0f-41f2-a901-ffc97c474051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:17.318585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d46761-ef0f-41f2-a901-ffc97c474051 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:17.318964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d46761-ef0f-41f2-a901-ffc97c474051 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:17.319253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d46761-ef0f-41f2-a901-ffc97c474051 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:17.319971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d46761-ef0f-41f2-a901-ffc97c474051 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:17.320282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d46761-ef0f-41f2-a901-ffc97c474051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:17.322261 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d46761-ef0f-41f2-a901-ffc97c474051 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:17.326876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:17.335857 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 468/926] 10.222.0.43 () {56 vars in 1126 bytes} [Mon Jul 4 07:51:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:17.338007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d46761-ef0f-41f2-a901-ffc97c474051 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:17.342958 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd97538-231f-4ceb-a06b-8fcd68f9873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:17.343981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd97538-231f-4ceb-a06b-8fcd68f9873b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:17.344501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd97538-231f-4ceb-a06b-8fcd68f9873b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:17.344987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd97538-231f-4ceb-a06b-8fcd68f9873b None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:17.345943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd97538-231f-4ceb-a06b-8fcd68f9873b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:17.346465 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd97538-231f-4ceb-a06b-8fcd68f9873b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:17.348060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd97538-231f-4ceb-a06b-8fcd68f9873b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:17.356596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 459/927] 10.222.0.43 () {56 vars in 1122 bytes} [Mon Jul 4 07:51:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:17.362327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd97538-231f-4ceb-a06b-8fcd68f9873b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:17.426016 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:17.491468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd97538-231f-4ceb-a06b-8fcd68f9873b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:17.502019 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d46761-ef0f-41f2-a901-ffc97c474051 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:17.540635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef47f7f-57c4-49e5-8730-aef7743b4f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:17.542270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef47f7f-57c4-49e5-8730-aef7743b4f5f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:17.542906 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef47f7f-57c4-49e5-8730-aef7743b4f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:17.543457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef47f7f-57c4-49e5-8730-aef7743b4f5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:17.544549 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef47f7f-57c4-49e5-8730-aef7743b4f5f None admin] RBAC: Adding query filter params (interface=public, service_id=ee1faa6056574eaaaecfeaaa8ab57bcc, region_id=RegionOne) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:17.545066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef47f7f-57c4-49e5-8730-aef7743b4f5f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:17.547684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef47f7f-57c4-49e5-8730-aef7743b4f5f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:17.560227 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 460/928] 10.222.0.43 () {56 vars in 1251 bytes} [Mon Jul 4 07:51:17 2022] GET /identity/v3/endpoints?service_id=ee1faa6056574eaaaecfeaaa8ab57bcc&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:17.562374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef47f7f-57c4-49e5-8730-aef7743b4f5f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:17.563086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf6a7b-f2cb-467d-88cd-17c97754cf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:17.564422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf6a7b-f2cb-467d-88cd-17c97754cf4a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:17.564962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf6a7b-f2cb-467d-88cd-17c97754cf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:17.565499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf6a7b-f2cb-467d-88cd-17c97754cf4a None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:17.566503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf6a7b-f2cb-467d-88cd-17c97754cf4a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:17.567166 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf6a7b-f2cb-467d-88cd-17c97754cf4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:17.569101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf6a7b-f2cb-467d-88cd-17c97754cf4a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:17.576090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 469/929] 10.222.0.43 () {56 vars in 1126 bytes} [Mon Jul 4 07:51:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:17.587403 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf6a7b-f2cb-467d-88cd-17c97754cf4a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:17.640678 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:17.716679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:17.718180 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef47f7f-57c4-49e5-8730-aef7743b4f5f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:17.731752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf6a7b-f2cb-467d-88cd-17c97754cf4a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:17.768938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619d77a-5fc3-40e8-8159-3dca0beba0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:17.770160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619d77a-5fc3-40e8-8159-3dca0beba0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:17.770597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619d77a-5fc3-40e8-8159-3dca0beba0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:17.771020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619d77a-5fc3-40e8-8159-3dca0beba0f4 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:17.771623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78dbefc-d1a9-4ed6-8afe-8cd97b3f33a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:17.772185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619d77a-5fc3-40e8-8159-3dca0beba0f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:17.772652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dbefc-d1a9-4ed6-8afe-8cd97b3f33a2 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:17.773028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dbefc-d1a9-4ed6-8afe-8cd97b3f33a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:17.773391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dbefc-d1a9-4ed6-8afe-8cd97b3f33a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:17.774335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78dbefc-d1a9-4ed6-8afe-8cd97b3f33a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:17.774840 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619d77a-5fc3-40e8-8159-3dca0beba0f4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:17.775996 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78dbefc-d1a9-4ed6-8afe-8cd97b3f33a2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:17.783169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 461/930] 10.222.0.43 () {56 vars in 1087 bytes} [Mon Jul 4 07:51:17 2022] GET /identity/v3/services => generated 1497 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 07:51:17.807356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 470/931] 10.222.0.43 () {60 vars in 1144 bytes} [Mon Jul 4 07:51:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:18.195185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861c2ec-3087-4d9e-a6e0-64f4fc3e3337 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:18.195551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861c2ec-3087-4d9e-a6e0-64f4fc3e3337 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:18.195750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861c2ec-3087-4d9e-a6e0-64f4fc3e3337 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:18.197368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 462/932] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:18.203557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054dfe0-fb0b-4a93-95b9-032133239fc0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:18.204205 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054dfe0-fb0b-4a93-95b9-032133239fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:18.204691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054dfe0-fb0b-4a93-95b9-032133239fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:18.224036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3054dfe0-fb0b-4a93-95b9-032133239fc0 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:18.225907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3054dfe0-fb0b-4a93-95b9-032133239fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:18.305813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 471/933] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:18.334676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861c2ec-3087-4d9e-a6e0-64f4fc3e3337 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:18.449647 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861c2ec-3087-4d9e-a6e0-64f4fc3e3337 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:18.467472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:18.486388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff518c-d364-47b7-8a95-c48a7d2e960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:18.487062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff518c-d364-47b7-8a95-c48a7d2e960c None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:18.487340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff518c-d364-47b7-8a95-c48a7d2e960c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:18.487617 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff518c-d364-47b7-8a95-c48a7d2e960c None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:18.488401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff518c-d364-47b7-8a95-c48a7d2e960c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:18.489934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff518c-d364-47b7-8a95-c48a7d2e960c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:18.491368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: WARNING keystone.server.flask.common [None req-7bff518c-d364-47b7-8a95-c48a7d2e960c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 04 07:51:18.511176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.sql.core [None req-7bff518c-d364-47b7-8a95-c48a7d2e960c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 07:51:18.511176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 04 07:51:18.511176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [parameters: {'id': '5682ada600e549b48993bef9d6807e29', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 04 07:51:18.511176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80144) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7bff518c-d364-47b7-8a95-c48a7d2e960c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5682ada600e549b48993bef9d6807e29.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5682ada600e549b48993bef9d6807e29. Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application conn.query(q) Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 07:51:18.532272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result.read() Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 04 07:51:18.533197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application next(self.gen) Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application yield resource Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application next(self.gen) Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise self.value Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application session.commit() Jul 04 07:51:18.534239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.session.flush() Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application compat.raise_( Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 07:51:18.535164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 07:51:18.536109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application conn.query(q) Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result.read() Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 07:51:18.537024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [parameters: {'id': '5682ada600e549b48993bef9d6807e29', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:18.538182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:18.539004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:18.540046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:18.540046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 04 07:51:18.540046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 04 07:51:18.540046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:18.540046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:18.540046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 04 07:51:18.540046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 04 07:51:18.540046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5682ada600e549b48993bef9d6807e29. Jul 04 07:51:18.540046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:18.540046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 463/934] 10.222.0.43 () {60 vars in 1140 bytes} [Mon Jul 4 07:51:18 2022] POST /identity/v3/projects => generated 262 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:18.542026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054dfe0-fb0b-4a93-95b9-032133239fc0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:18.623071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:18.663097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054dfe0-fb0b-4a93-95b9-032133239fc0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:18.699603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9b846-0744-46ae-be6b-c740460b6afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:18.700483 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9b846-0744-46ae-be6b-c740460b6afe None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:18.700706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9b846-0744-46ae-be6b-c740460b6afe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:18.700983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9b846-0744-46ae-be6b-c740460b6afe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:18.708598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9b846-0744-46ae-be6b-c740460b6afe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:18.710232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9b846-0744-46ae-be6b-c740460b6afe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c9b9b846-0744-46ae-be6b-c740460b6afe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:18.724880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:18.725919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:18.727600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:18.727600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:18.727600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:18.727600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:18.727600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:18.727600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:18.727600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:18.727600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:18.727600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 472/935] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:18.735253 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff518c-d364-47b7-8a95-c48a7d2e960c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:18.859885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff518c-d364-47b7-8a95-c48a7d2e960c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:18.905768 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03257c5d-d46e-4591-b0bf-a06ed14353bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:18.906495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03257c5d-d46e-4591-b0bf-a06ed14353bf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:18.906774 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03257c5d-d46e-4591-b0bf-a06ed14353bf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:18.907059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03257c5d-d46e-4591-b0bf-a06ed14353bf None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:18.907938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03257c5d-d46e-4591-b0bf-a06ed14353bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:18.908241 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03257c5d-d46e-4591-b0bf-a06ed14353bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:18.909933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03257c5d-d46e-4591-b0bf-a06ed14353bf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:18.933024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 464/936] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:51:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:19.192862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa9f76-df9a-4b55-8e9b-5d24d4c71dcf None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:19.193225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa9f76-df9a-4b55-8e9b-5d24d4c71dcf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:19.193411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa9f76-df9a-4b55-8e9b-5d24d4c71dcf None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:19.195389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 473/937] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:19.201414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a7d7a-fe3a-4db1-824c-47a80645beab None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:19.201752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a7d7a-fe3a-4db1-824c-47a80645beab None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:19.202027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a7d7a-fe3a-4db1-824c-47a80645beab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:19.220863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4d1a7d7a-fe3a-4db1-824c-47a80645beab None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:19.222447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4d1a7d7a-fe3a-4db1-824c-47a80645beab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:19.253996 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:19.299682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 465/938] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:19.322659 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa9f76-df9a-4b55-8e9b-5d24d4c71dcf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:19.434350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa9f76-df9a-4b55-8e9b-5d24d4c71dcf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:19.465793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:19.472514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee66944-ff7e-4810-bbf8-e286988260d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:19.473555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66944-ff7e-4810-bbf8-e286988260d5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:19.473882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66944-ff7e-4810-bbf8-e286988260d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:19.474159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66944-ff7e-4810-bbf8-e286988260d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:19.485723 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee66944-ff7e-4810-bbf8-e286988260d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-aee66944-ff7e-4810-bbf8-e286988260d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:19.494270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:19.495292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:19.496559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 474/939] 10.222.0.43 () {56 vars in 1120 bytes} [Mon Jul 4 07:51:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:19.502156 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a7d7a-fe3a-4db1-824c-47a80645beab None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:19.611882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a7d7a-fe3a-4db1-824c-47a80645beab None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:19.647554 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9b8f5-f59c-4261-97a2-411b499490aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:19.648318 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9b8f5-f59c-4261-97a2-411b499490aa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:19.648598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9b8f5-f59c-4261-97a2-411b499490aa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:19.648879 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9b8f5-f59c-4261-97a2-411b499490aa None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:19.649644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9b8f5-f59c-4261-97a2-411b499490aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:19.649963 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9b8f5-f59c-4261-97a2-411b499490aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:19.651425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9b8f5-f59c-4261-97a2-411b499490aa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:19.660786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 466/940] 10.222.0.43 () {56 vars in 1115 bytes} [Mon Jul 4 07:51:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:19.666672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee66944-ff7e-4810-bbf8-e286988260d5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:19.776443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee66944-ff7e-4810-bbf8-e286988260d5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:19.813222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781ade1-4299-4158-a547-00ba58c811e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:19.814123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781ade1-4299-4158-a547-00ba58c811e7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:19.814416 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781ade1-4299-4158-a547-00ba58c811e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:19.814807 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781ade1-4299-4158-a547-00ba58c811e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:19.826699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781ade1-4299-4158-a547-00ba58c811e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:19.828189 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781ade1-4299-4158-a547-00ba58c811e7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c781ade1-4299-4158-a547-00ba58c811e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:19.846744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:19.849035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:19.851581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:19.851581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:19.851581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:19.851581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:19.851581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 475/941] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:19.855110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9b8f5-f59c-4261-97a2-411b499490aa None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:19.888591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:19.972770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9b8f5-f59c-4261-97a2-411b499490aa None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:20.006555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f263a57-9d0a-4657-9c59-c467a4aaf39f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.007201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f263a57-9d0a-4657-9c59-c467a4aaf39f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.007688 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f263a57-9d0a-4657-9c59-c467a4aaf39f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.009455 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 476/942] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:20.010482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06adefa4-7134-4d01-97c8-7bfbbae7a44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:20.011154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adefa4-7134-4d01-97c8-7bfbbae7a44a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.011436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adefa4-7134-4d01-97c8-7bfbbae7a44a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.011709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adefa4-7134-4d01-97c8-7bfbbae7a44a None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.012459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06adefa4-7134-4d01-97c8-7bfbbae7a44a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:20.012955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06adefa4-7134-4d01-97c8-7bfbbae7a44a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:20.014367 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06adefa4-7134-4d01-97c8-7bfbbae7a44a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:20.014659 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8081e7-eaee-4c42-b520-c5b4d14a2176 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.015136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8081e7-eaee-4c42-b520-c5b4d14a2176 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.015616 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8081e7-eaee-4c42-b520-c5b4d14a2176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.027354 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 467/943] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:20.029359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82456df1-025b-4f59-9ee0-a27302824601 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.029729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82456df1-025b-4f59-9ee0-a27302824601 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.030004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82456df1-025b-4f59-9ee0-a27302824601 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.031475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 468/944] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:20.033965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82456df1-025b-4f59-9ee0-a27302824601 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:20.038778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9c8081e7-eaee-4c42-b520-c5b4d14a2176 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:20.039981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9c8081e7-eaee-4c42-b520-c5b4d14a2176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:20.078523 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:20.118976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 477/945] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:20.120926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5a7cc-bb67-4185-953c-42892a7f060c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.121215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5a7cc-bb67-4185-953c-42892a7f060c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.121491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5a7cc-bb67-4185-953c-42892a7f060c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.136224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:20.148447 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-45f5a7cc-bb67-4185-953c-42892a7f060c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:20.149692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-45f5a7cc-bb67-4185-953c-42892a7f060c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:20.154844 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82456df1-025b-4f59-9ee0-a27302824601 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:20.196511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b58aeb-18a9-4813-a2e9-1ee3ea2d949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:20.197371 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b58aeb-18a9-4813-a2e9-1ee3ea2d949e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.198139 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b58aeb-18a9-4813-a2e9-1ee3ea2d949e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.198430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b58aeb-18a9-4813-a2e9-1ee3ea2d949e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.207073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b58aeb-18a9-4813-a2e9-1ee3ea2d949e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:20.208689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b58aeb-18a9-4813-a2e9-1ee3ea2d949e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-d7b58aeb-18a9-4813-a2e9-1ee3ea2d949e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:51:20.225711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:20.226775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:20.227765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:20.227765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:20.227765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:51:20.227765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:51:20.227765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:51:20.227765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:51:20.227765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:51:20.227765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:51:20.227765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 07:51:20.227765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:20.227765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 469/946] 10.222.0.43 () {56 vars in 1117 bytes} [Mon Jul 4 07:51:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:20.228374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b58aeb-18a9-4813-a2e9-1ee3ea2d949e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:20.234626 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 478/947] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:20.238557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5a7cc-bb67-4185-953c-42892a7f060c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:20.347523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b58aeb-18a9-4813-a2e9-1ee3ea2d949e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:20.354554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5a7cc-bb67-4185-953c-42892a7f060c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:20.356232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:20.384837 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96cfee-9348-4308-8373-dbf370efc0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:20.385683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96cfee-9348-4308-8373-dbf370efc0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.385968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96cfee-9348-4308-8373-dbf370efc0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.386255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96cfee-9348-4308-8373-dbf370efc0f4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.386897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96cfee-9348-4308-8373-dbf370efc0f4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:20.388268 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96cfee-9348-4308-8373-dbf370efc0f4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:20.392759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a6703-a9b5-48f6-85fb-89424b37e89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:20.393598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a6703-a9b5-48f6-85fb-89424b37e89e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.394044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a6703-a9b5-48f6-85fb-89424b37e89e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.394470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a6703-a9b5-48f6-85fb-89424b37e89e None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.395448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a6703-a9b5-48f6-85fb-89424b37e89e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:20.395903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a6703-a9b5-48f6-85fb-89424b37e89e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:20.397552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a6703-a9b5-48f6-85fb-89424b37e89e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-af96cfee-9348-4308-8373-dbf370efc0f4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:20.400532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:20.401527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:20.402475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 470/948] 10.222.0.43 () {56 vars in 1126 bytes} [Mon Jul 4 07:51:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:20.403281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96cfee-9348-4308-8373-dbf370efc0f4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:20.420194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 479/949] 10.222.0.43 () {56 vars in 1149 bytes} [Mon Jul 4 07:51:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:20.421060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a6703-a9b5-48f6-85fb-89424b37e89e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:20.425001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:20.522037 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96cfee-9348-4308-8373-dbf370efc0f4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:20.528430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a6703-a9b5-48f6-85fb-89424b37e89e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:20.558059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604d2df-6358-4dd7-bab8-17e9c3526be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:20.558876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604d2df-6358-4dd7-bab8-17e9c3526be3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.559157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604d2df-6358-4dd7-bab8-17e9c3526be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.559428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604d2df-6358-4dd7-bab8-17e9c3526be3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.560079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a604d2df-6358-4dd7-bab8-17e9c3526be3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:20.561482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a604d2df-6358-4dd7-bab8-17e9c3526be3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:20.564390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351e414-8437-4908-81ed-eb4d945a36ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:20.565072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351e414-8437-4908-81ed-eb4d945a36ed None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.565349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351e414-8437-4908-81ed-eb4d945a36ed None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.565650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351e414-8437-4908-81ed-eb4d945a36ed None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.566358 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351e414-8437-4908-81ed-eb4d945a36ed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:20.566665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351e414-8437-4908-81ed-eb4d945a36ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:20.568034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351e414-8437-4908-81ed-eb4d945a36ed None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:20.577418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 480/950] 10.222.0.43 () {56 vars in 1122 bytes} [Mon Jul 4 07:51:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:20.577418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351e414-8437-4908-81ed-eb4d945a36ed None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a604d2df-6358-4dd7-bab8-17e9c3526be3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:20.577703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:20.579026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:20.579955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 471/951] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:20.580831 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604d2df-6358-4dd7-bab8-17e9c3526be3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:20.608989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:20.664561 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:20.690598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351e414-8437-4908-81ed-eb4d945a36ed None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:20.693564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604d2df-6358-4dd7-bab8-17e9c3526be3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:20.728180 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483404b5-5729-474a-ae6f-643a41e1172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:20.729014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483404b5-5729-474a-ae6f-643a41e1172e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.729299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483404b5-5729-474a-ae6f-643a41e1172e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.729574 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483404b5-5729-474a-ae6f-643a41e1172e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.731351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c02be-6216-4a08-976f-11ef4a7ae3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:20.732059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c02be-6216-4a08-976f-11ef4a7ae3de None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.732341 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c02be-6216-4a08-976f-11ef4a7ae3de None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.732807 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c02be-6216-4a08-976f-11ef4a7ae3de None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.733329 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c02be-6216-4a08-976f-11ef4a7ae3de None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:20.733654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c02be-6216-4a08-976f-11ef4a7ae3de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:20.735050 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c02be-6216-4a08-976f-11ef4a7ae3de None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:20.745251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 472/952] 10.222.0.43 () {56 vars in 1122 bytes} [Mon Jul 4 07:51:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:20.745430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483404b5-5729-474a-ae6f-643a41e1172e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:20.745542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c02be-6216-4a08-976f-11ef4a7ae3de None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:20.745645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483404b5-5729-474a-ae6f-643a41e1172e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-483404b5-5729-474a-ae6f-643a41e1172e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:20.766878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:20.767865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:20.768776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:20.768776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:20.768776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:20.768776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:20.768776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 481/953] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:20.769497 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483404b5-5729-474a-ae6f-643a41e1172e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:20.849423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:20.861645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c02be-6216-4a08-976f-11ef4a7ae3de None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:20.875666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483404b5-5729-474a-ae6f-643a41e1172e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:20.914460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fec26-3ae3-4c41-a5d2-c4441adc3d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:20.915124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fec26-3ae3-4c41-a5d2-c4441adc3d6a None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.915410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fec26-3ae3-4c41-a5d2-c4441adc3d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.915695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fec26-3ae3-4c41-a5d2-c4441adc3d6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.916428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0fec26-3ae3-4c41-a5d2-c4441adc3d6a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:20.917847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0fec26-3ae3-4c41-a5d2-c4441adc3d6a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:20.923129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf4648-85f9-4cb7-a3e7-98625f6f50eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:20.924203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf4648-85f9-4cb7-a3e7-98625f6f50eb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:20.924647 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf4648-85f9-4cb7-a3e7-98625f6f50eb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:20.925061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf4648-85f9-4cb7-a3e7-98625f6f50eb None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:20.926154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf4648-85f9-4cb7-a3e7-98625f6f50eb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:20.927087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf4648-85f9-4cb7-a3e7-98625f6f50eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:20.928911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf4648-85f9-4cb7-a3e7-98625f6f50eb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:20.935145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 473/954] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:51:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:20.936053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf4648-85f9-4cb7-a3e7-98625f6f50eb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:20.942582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:20.949645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 482/955] 10.222.0.43 () {60 vars in 1144 bytes} [Mon Jul 4 07:51:20 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:20.949645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fec26-3ae3-4c41-a5d2-c4441adc3d6a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:21.076880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fec26-3ae3-4c41-a5d2-c4441adc3d6a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:21.085488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf4648-85f9-4cb7-a3e7-98625f6f50eb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:21.110408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:21.122549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15874a9-dd3a-40d0-92f3-51b9ae5a0074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:21.123901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15874a9-dd3a-40d0-92f3-51b9ae5a0074 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:21.123901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15874a9-dd3a-40d0-92f3-51b9ae5a0074 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:21.124865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15874a9-dd3a-40d0-92f3-51b9ae5a0074 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:21.125607 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15874a9-dd3a-40d0-92f3-51b9ae5a0074 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:21.125932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15874a9-dd3a-40d0-92f3-51b9ae5a0074 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:21.127661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15874a9-dd3a-40d0-92f3-51b9ae5a0074 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:21.131098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c204c9-b07c-4262-8c78-3eb023253170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:21.132008 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c204c9-b07c-4262-8c78-3eb023253170 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:21.132292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c204c9-b07c-4262-8c78-3eb023253170 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:21.132565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c204c9-b07c-4262-8c78-3eb023253170 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:21.133294 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c204c9-b07c-4262-8c78-3eb023253170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:21.133601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c204c9-b07c-4262-8c78-3eb023253170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:21.134253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 483/956] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:51:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:21.135094 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c204c9-b07c-4262-8c78-3eb023253170 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:21.135563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d9ac7-642b-4769-8d29-fd3ff47c0291 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:21.135855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d9ac7-642b-4769-8d29-fd3ff47c0291 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:21.136136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d9ac7-642b-4769-8d29-fd3ff47c0291 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:21.137555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 484/957] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:21.143636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb707c8d-350d-4224-87be-e7e2ec0557d7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:21.143966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb707c8d-350d-4224-87be-e7e2ec0557d7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:21.144196 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb707c8d-350d-4224-87be-e7e2ec0557d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:21.153783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 474/958] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:21.160357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c204c9-b07c-4262-8c78-3eb023253170 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:21.164068 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fb707c8d-350d-4224-87be-e7e2ec0557d7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:21.165211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fb707c8d-350d-4224-87be-e7e2ec0557d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:21.265725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 485/959] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:21.273476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:21.299152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb707c8d-350d-4224-87be-e7e2ec0557d7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:21.338563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c204c9-b07c-4262-8c78-3eb023253170 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:21.382460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9903c-9b81-4d00-b97b-6be9d145a868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:21.384233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9903c-9b81-4d00-b97b-6be9d145a868 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:21.384716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9903c-9b81-4d00-b97b-6be9d145a868 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:21.385374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9903c-9b81-4d00-b97b-6be9d145a868 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:21.394966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9903c-9b81-4d00-b97b-6be9d145a868 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:21.397405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9903c-9b81-4d00-b97b-6be9d145a868 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-cec9903c-9b81-4d00-b97b-6be9d145a868 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:21.415137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:21.416168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:21.417098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:21.417098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:21.417098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:21.417098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:21.417098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:21.417098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:21.417098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:21.417098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:21.423770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 475/960] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:21.427321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9903c-9b81-4d00-b97b-6be9d145a868 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:21.461802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb707c8d-350d-4224-87be-e7e2ec0557d7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:21.494867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:21.513992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423d1a7-0474-407c-9f37-150828bede03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:21.514693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423d1a7-0474-407c-9f37-150828bede03 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:21.515036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423d1a7-0474-407c-9f37-150828bede03 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:21.515320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423d1a7-0474-407c-9f37-150828bede03 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:21.516036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423d1a7-0474-407c-9f37-150828bede03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:21.517883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423d1a7-0474-407c-9f37-150828bede03 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:21.519930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: WARNING keystone.server.flask.common [None req-9423d1a7-0474-407c-9f37-150828bede03 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 04 07:51:21.537757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.sql.core [None req-9423d1a7-0474-407c-9f37-150828bede03 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 07:51:21.537757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 04 07:51:21.537757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [parameters: {'user_id': 'f66e108f253946f7a9af152273ab45a3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 04 07:51:21.537757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80143) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9423d1a7-0474-407c-9f37-150828bede03 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application conn.query(q) Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result.read() Jul 04 07:51:21.551316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 07:51:21.552285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application next(self.gen) Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application yield resource Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application next(self.gen) Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise self.value Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application session.commit() Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 07:51:21.553354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.session.flush() Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application compat.raise_( Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 07:51:21.554383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 07:51:21.556155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application conn.query(q) Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result.read() Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 07:51:21.557539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f66e108f253946f7a9af152273ab45a3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:21.558514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 04 07:51:21.560595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 04 07:51:21.562764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 04 07:51:21.562764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 04 07:51:21.562764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 07:51:21.562764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 07:51:21.562764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 04 07:51:21.562764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:21.566384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9903c-9b81-4d00-b97b-6be9d145a868 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:21.570767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 486/961] 10.222.0.43 () {60 vars in 1131 bytes} [Mon Jul 4 07:51:21 2022] POST /identity/v3/users => generated 160 bytes in 268 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:21.572745 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423d1a7-0474-407c-9f37-150828bede03 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:21.593970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:21.608195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecaee5-c196-4dd1-b4d7-1bd0d47b1aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:21.608898 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecaee5-c196-4dd1-b4d7-1bd0d47b1aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:21.609306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecaee5-c196-4dd1-b4d7-1bd0d47b1aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:21.609455 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecaee5-c196-4dd1-b4d7-1bd0d47b1aa0 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:21.610286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecaee5-c196-4dd1-b4d7-1bd0d47b1aa0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:21.610658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecaee5-c196-4dd1-b4d7-1bd0d47b1aa0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:21.612248 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecaee5-c196-4dd1-b4d7-1bd0d47b1aa0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:21.631115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 476/962] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:51:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:21.634852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecaee5-c196-4dd1-b4d7-1bd0d47b1aa0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:21.719654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423d1a7-0474-407c-9f37-150828bede03 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:21.761269 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b560554-6c6c-4e44-a3ed-2a7bf32aec4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:21.762237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b560554-6c6c-4e44-a3ed-2a7bf32aec4c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:21.762522 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b560554-6c6c-4e44-a3ed-2a7bf32aec4c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:21.762800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b560554-6c6c-4e44-a3ed-2a7bf32aec4c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:21.772010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b560554-6c6c-4e44-a3ed-2a7bf32aec4c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:21.774183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b560554-6c6c-4e44-a3ed-2a7bf32aec4c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:21.782532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecaee5-c196-4dd1-b4d7-1bd0d47b1aa0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-2b560554-6c6c-4e44-a3ed-2a7bf32aec4c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:51:21.791159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:21.792211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:21.793201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:21.793201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:21.793201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:51:21.793201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:51:21.793201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:51:21.793201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:51:21.793201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:51:21.793201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:51:21.793201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 07:51:21.793201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:21.797951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 487/963] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:51:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:21.799623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b560554-6c6c-4e44-a3ed-2a7bf32aec4c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:21.829593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46efc30-bc4e-429e-8bbc-8bac41e21625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:21.831087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46efc30-bc4e-429e-8bbc-8bac41e21625 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:21.831929 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46efc30-bc4e-429e-8bbc-8bac41e21625 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:21.832613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46efc30-bc4e-429e-8bbc-8bac41e21625 None admin] PATH_INFO: `/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/80a45a063c0e4f1e9b2156fb2f4cd8b8/roles/41ba7c35817d4f368d33695911c9295e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:21.836223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46efc30-bc4e-429e-8bbc-8bac41e21625 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f2d85c9e3d14bdabfb63fe505d83714, user_id=80a45a063c0e4f1e9b2156fb2f4cd8b8, role_id=41ba7c35817d4f368d33695911c9295e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:21.838346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46efc30-bc4e-429e-8bbc-8bac41e21625 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:21.855830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:21.865904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 477/964] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:51:21 2022] PUT /identity/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/80a45a063c0e4f1e9b2156fb2f4cd8b8/roles/41ba7c35817d4f368d33695911c9295e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 07:51:21.935499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b560554-6c6c-4e44-a3ed-2a7bf32aec4c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:21.977632 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a60da-068e-4f5b-a23e-79219eccd354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:21.978372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a60da-068e-4f5b-a23e-79219eccd354 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:21.978654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a60da-068e-4f5b-a23e-79219eccd354 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:21.978931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a60da-068e-4f5b-a23e-79219eccd354 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:21.979790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a60da-068e-4f5b-a23e-79219eccd354 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:21.980106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a60da-068e-4f5b-a23e-79219eccd354 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:21.981786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a60da-068e-4f5b-a23e-79219eccd354 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:22.029959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 488/965] 10.222.0.43 () {56 vars in 1101 bytes} [Mon Jul 4 07:51:21 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:23.119645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a397e-7658-4ed5-8130-a379a05d2930 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:23.120155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a397e-7658-4ed5-8130-a379a05d2930 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:23.120275 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a397e-7658-4ed5-8130-a379a05d2930 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:23.121578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 478/966] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:23.127171 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796d8a4-d7b8-4084-af58-56f251ebbd4f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:23.127618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796d8a4-d7b8-4084-af58-56f251ebbd4f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:23.127695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796d8a4-d7b8-4084-af58-56f251ebbd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:23.145572 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5796d8a4-d7b8-4084-af58-56f251ebbd4f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:23.147169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5796d8a4-d7b8-4084-af58-56f251ebbd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:23.220315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 489/967] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:23.242966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a397e-7658-4ed5-8130-a379a05d2930 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:23.243955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0be9c9-4ca7-43fc-b9f5-1429cda5e490 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:23.244316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0be9c9-4ca7-43fc-b9f5-1429cda5e490 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:23.244607 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0be9c9-4ca7-43fc-b9f5-1429cda5e490 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:23.246161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 490/968] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:23.251074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1383d4-73b5-4596-8878-159903f35984 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:23.251365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1383d4-73b5-4596-8878-159903f35984 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:23.251648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1383d4-73b5-4596-8878-159903f35984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:23.269872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ab1383d4-73b5-4596-8878-159903f35984 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:23.271099 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ab1383d4-73b5-4596-8878-159903f35984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:23.289344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:23.301335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:23.346943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 491/969] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:23.356707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a397e-7658-4ed5-8130-a379a05d2930 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:23.372530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1383d4-73b5-4596-8878-159903f35984 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:23.398617 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba769dca-f431-4034-8e79-75d313822ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:23.399435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba769dca-f431-4034-8e79-75d313822ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:23.399712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba769dca-f431-4034-8e79-75d313822ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:23.400000 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba769dca-f431-4034-8e79-75d313822ee4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:23.400635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba769dca-f431-4034-8e79-75d313822ee4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:23.402074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba769dca-f431-4034-8e79-75d313822ee4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ba769dca-f431-4034-8e79-75d313822ee4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:23.415160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:23.416330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:23.417218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 479/970] 10.222.0.43 () {56 vars in 1126 bytes} [Mon Jul 4 07:51:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:23.422397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba769dca-f431-4034-8e79-75d313822ee4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:23.497161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1383d4-73b5-4596-8878-159903f35984 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:23.539334 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d4fc2-5262-4562-8cc2-f0846fe3aae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:23.540131 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d4fc2-5262-4562-8cc2-f0846fe3aae2 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:23.540477 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d4fc2-5262-4562-8cc2-f0846fe3aae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:23.540820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d4fc2-5262-4562-8cc2-f0846fe3aae2 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:23.541637 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d4fc2-5262-4562-8cc2-f0846fe3aae2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:23.543131 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d4fc2-5262-4562-8cc2-f0846fe3aae2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:23.557602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba769dca-f431-4034-8e79-75d313822ee4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:23.558224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 492/971] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:51:23 2022] POST /identity/v3/services => generated 242 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 07:51:23.574208 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:23.641061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d626b-7fae-4dc6-875c-88bf6210e912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:23.642171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d626b-7fae-4dc6-875c-88bf6210e912 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:23.642652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d626b-7fae-4dc6-875c-88bf6210e912 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:23.643147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d626b-7fae-4dc6-875c-88bf6210e912 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:23.644416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d626b-7fae-4dc6-875c-88bf6210e912 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:23.645033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d626b-7fae-4dc6-875c-88bf6210e912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:23.648237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d626b-7fae-4dc6-875c-88bf6210e912 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:23.663450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 480/972] 10.222.0.43 () {56 vars in 1122 bytes} [Mon Jul 4 07:51:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:23.664908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d4fc2-5262-4562-8cc2-f0846fe3aae2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:23.693018 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:23.780783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d4fc2-5262-4562-8cc2-f0846fe3aae2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:23.818760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f81072-08e0-49bc-9913-c56529762cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:23.819435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f81072-08e0-49bc-9913-c56529762cfa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:23.819714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f81072-08e0-49bc-9913-c56529762cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:23.819987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f81072-08e0-49bc-9913-c56529762cfa None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:23.820774 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f81072-08e0-49bc-9913-c56529762cfa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:23.821081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f81072-08e0-49bc-9913-c56529762cfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:23.822507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f81072-08e0-49bc-9913-c56529762cfa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:23.833071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 493/973] 10.222.0.43 () {56 vars in 1122 bytes} [Mon Jul 4 07:51:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:23.834152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d626b-7fae-4dc6-875c-88bf6210e912 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:23.938165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d626b-7fae-4dc6-875c-88bf6210e912 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:23.974324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88956f-4234-4178-89ba-41b2394d409d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:23.975077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88956f-4234-4178-89ba-41b2394d409d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:23.975437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88956f-4234-4178-89ba-41b2394d409d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:23.975880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88956f-4234-4178-89ba-41b2394d409d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:23.976917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88956f-4234-4178-89ba-41b2394d409d None admin] RBAC: Adding query filter params (interface=admin, service_id=ee1faa6056574eaaaecfeaaa8ab57bcc, region_id=RegionOne) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:23.977228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88956f-4234-4178-89ba-41b2394d409d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:23.978675 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88956f-4234-4178-89ba-41b2394d409d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:23.988495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 481/974] 10.222.0.43 () {56 vars in 1249 bytes} [Mon Jul 4 07:51:23 2022] GET /identity/v3/endpoints?service_id=ee1faa6056574eaaaecfeaaa8ab57bcc&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:23.989803 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f81072-08e0-49bc-9913-c56529762cfa None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:24.007677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c2600-d9d7-408d-b746-7196577c41f9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:24.007967 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c2600-d9d7-408d-b746-7196577c41f9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:24.008243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c2600-d9d7-408d-b746-7196577c41f9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:24.009681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 482/975] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:24.015721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9adff-072d-481a-b56d-848c872a8386 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:24.016005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9adff-072d-481a-b56d-848c872a8386 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:24.016279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9adff-072d-481a-b56d-848c872a8386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:24.033338 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-32a9adff-072d-481a-b56d-848c872a8386 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:24.034603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-32a9adff-072d-481a-b56d-848c872a8386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:24.058557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:24.082157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:24.101102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f81072-08e0-49bc-9913-c56529762cfa None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:24.107204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 483/976] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:24.130932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9adff-072d-481a-b56d-848c872a8386 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:24.140559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601937e-ff08-402b-86c7-4dabd9714766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:24.141346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601937e-ff08-402b-86c7-4dabd9714766 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:24.141725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601937e-ff08-402b-86c7-4dabd9714766 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:24.142004 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601937e-ff08-402b-86c7-4dabd9714766 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:24.142690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601937e-ff08-402b-86c7-4dabd9714766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:24.144080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601937e-ff08-402b-86c7-4dabd9714766 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:24.150803 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 494/977] 10.222.0.43 () {56 vars in 1087 bytes} [Mon Jul 4 07:51:23 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 07:51:24.253671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9adff-072d-481a-b56d-848c872a8386 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:24.298557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64702597-fd81-4843-bc21-81c8db7a48b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:24.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64702597-fd81-4843-bc21-81c8db7a48b4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:24.299706 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64702597-fd81-4843-bc21-81c8db7a48b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:24.299987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64702597-fd81-4843-bc21-81c8db7a48b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:24.312770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64702597-fd81-4843-bc21-81c8db7a48b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-64702597-fd81-4843-bc21-81c8db7a48b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:24.321551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:24.322557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:24.323439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 484/978] 10.222.0.43 () {56 vars in 1120 bytes} [Mon Jul 4 07:51:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:24.329078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601937e-ff08-402b-86c7-4dabd9714766 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:24.441253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601937e-ff08-402b-86c7-4dabd9714766 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:24.478504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc6bab-bdf4-4660-bcde-8ec31f7f33e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:24.479179 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc6bab-bdf4-4660-bcde-8ec31f7f33e2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:24.479516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc6bab-bdf4-4660-bcde-8ec31f7f33e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:24.479792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc6bab-bdf4-4660-bcde-8ec31f7f33e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:24.480612 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc6bab-bdf4-4660-bcde-8ec31f7f33e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:24.480923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc6bab-bdf4-4660-bcde-8ec31f7f33e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:24.482323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc6bab-bdf4-4660-bcde-8ec31f7f33e2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:24.496965 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 495/979] 10.222.0.43 () {56 vars in 1115 bytes} [Mon Jul 4 07:51:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:24.498663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64702597-fd81-4843-bc21-81c8db7a48b4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:24.523518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:24.606178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64702597-fd81-4843-bc21-81c8db7a48b4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:24.644123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd024202-6885-4162-80ea-5550ca5fe88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:24.644947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd024202-6885-4162-80ea-5550ca5fe88d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:24.645228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd024202-6885-4162-80ea-5550ca5fe88d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:24.645504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd024202-6885-4162-80ea-5550ca5fe88d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:24.657363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd024202-6885-4162-80ea-5550ca5fe88d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:24.658875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd024202-6885-4162-80ea-5550ca5fe88d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bd024202-6885-4162-80ea-5550ca5fe88d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:24.677383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:24.678649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:24.679762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:24.679762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:24.679762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:24.679762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:24.683455 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 485/980] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:24.684943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc6bab-bdf4-4660-bcde-8ec31f7f33e2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:24.702042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:24.800874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc6bab-bdf4-4660-bcde-8ec31f7f33e2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:24.837858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f968eb-7fb4-43dd-9c66-250a59ee27d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:24.838561 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f968eb-7fb4-43dd-9c66-250a59ee27d1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:24.838837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f968eb-7fb4-43dd-9c66-250a59ee27d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:24.839112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f968eb-7fb4-43dd-9c66-250a59ee27d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:24.839848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f968eb-7fb4-43dd-9c66-250a59ee27d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:24.840133 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f968eb-7fb4-43dd-9c66-250a59ee27d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:24.841540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f968eb-7fb4-43dd-9c66-250a59ee27d1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:24.854769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 496/981] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:24.860413 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd024202-6885-4162-80ea-5550ca5fe88d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:24.942609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:24.976878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd024202-6885-4162-80ea-5550ca5fe88d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:25.012925 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae9b0b-2780-4921-8a07-7602890d09d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:25.013814 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae9b0b-2780-4921-8a07-7602890d09d5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:25.014107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae9b0b-2780-4921-8a07-7602890d09d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:25.014384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae9b0b-2780-4921-8a07-7602890d09d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:25.022398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae9b0b-2780-4921-8a07-7602890d09d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:25.023905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae9b0b-2780-4921-8a07-7602890d09d5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-06ae9b0b-2780-4921-8a07-7602890d09d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:51:25.039102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:25.040176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:25.041191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:25.041191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:25.041191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:51:25.041191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:51:25.041191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:51:25.041191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:51:25.041191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:51:25.041191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:51:25.041191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 07:51:25.041191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:25.041191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 486/982] 10.222.0.43 () {56 vars in 1117 bytes} [Mon Jul 4 07:51:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:25.048263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f968eb-7fb4-43dd-9c66-250a59ee27d1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:25.139977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:25.174421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f968eb-7fb4-43dd-9c66-250a59ee27d1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:25.217653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ba410-34f0-4e0d-a166-8e58aeea7fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:25.218362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ba410-34f0-4e0d-a166-8e58aeea7fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:25.218638 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ba410-34f0-4e0d-a166-8e58aeea7fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:25.218913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ba410-34f0-4e0d-a166-8e58aeea7fa5 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:25.219763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ba410-34f0-4e0d-a166-8e58aeea7fa5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:25.220071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ba410-34f0-4e0d-a166-8e58aeea7fa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:25.221531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ba410-34f0-4e0d-a166-8e58aeea7fa5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:25.243884 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 497/983] 10.222.0.43 () {56 vars in 1149 bytes} [Mon Jul 4 07:51:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:25.249410 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae9b0b-2780-4921-8a07-7602890d09d5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:25.372105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae9b0b-2780-4921-8a07-7602890d09d5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:25.403276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:25.409644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7280f-ab52-4479-bf0d-ca2962cee0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:25.410326 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7280f-ab52-4479-bf0d-ca2962cee0cf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:25.410606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7280f-ab52-4479-bf0d-ca2962cee0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:25.410873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7280f-ab52-4479-bf0d-ca2962cee0cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:25.483150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7280f-ab52-4479-bf0d-ca2962cee0cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:25.484706 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7280f-ab52-4479-bf0d-ca2962cee0cf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:25.592039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 487/984] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:51:25 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:51:25.597595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ba410-34f0-4e0d-a166-8e58aeea7fa5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:25.639171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a1137-a5d0-4f84-95a6-f69e268fb673 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:25.639374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a1137-a5d0-4f84-95a6-f69e268fb673 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:25.639618 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a1137-a5d0-4f84-95a6-f69e268fb673 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:25.641032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 488/985] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:25.646987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d0560-3f1f-49d6-a603-91225fefe45c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:25.647343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d0560-3f1f-49d6-a603-91225fefe45c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:25.647623 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d0560-3f1f-49d6-a603-91225fefe45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:25.665299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-197d0560-3f1f-49d6-a603-91225fefe45c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:25.666575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-197d0560-3f1f-49d6-a603-91225fefe45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:25.701379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:25.711032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ba410-34f0-4e0d-a166-8e58aeea7fa5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:25.746051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 489/986] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:25.751535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:25.759154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63061bf8-2cc6-429d-a636-59d4bfbbb3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:25.760050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63061bf8-2cc6-429d-a636-59d4bfbbb3ab None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:25.760338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63061bf8-2cc6-429d-a636-59d4bfbbb3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:25.760611 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63061bf8-2cc6-429d-a636-59d4bfbbb3ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:25.769961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d0560-3f1f-49d6-a603-91225fefe45c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:25.773248 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63061bf8-2cc6-429d-a636-59d4bfbbb3ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:25.774810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63061bf8-2cc6-429d-a636-59d4bfbbb3ab None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-63061bf8-2cc6-429d-a636-59d4bfbbb3ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:25.794521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:25.795671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:25.796653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:25.796653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:25.796653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:25.796653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:25.796653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 498/987] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:25.802038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63061bf8-2cc6-429d-a636-59d4bfbbb3ab None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:25.889026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d0560-3f1f-49d6-a603-91225fefe45c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:25.920467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63061bf8-2cc6-429d-a636-59d4bfbbb3ab None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:25.927102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb0411-9dc9-462c-a4ed-e0f0f8eeabf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:25.927915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb0411-9dc9-462c-a4ed-e0f0f8eeabf8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:25.928229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb0411-9dc9-462c-a4ed-e0f0f8eeabf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:25.928991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb0411-9dc9-462c-a4ed-e0f0f8eeabf8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:25.929679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb0411-9dc9-462c-a4ed-e0f0f8eeabf8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:25.931406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb0411-9dc9-462c-a4ed-e0f0f8eeabf8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-17bb0411-9dc9-462c-a4ed-e0f0f8eeabf8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:25.944211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:25.945481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:25.946439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 490/988] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:25.955830 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb0411-9dc9-462c-a4ed-e0f0f8eeabf8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:25.962746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6f793-afe3-4bd0-8477-a1a49e40346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:25.963430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6f793-afe3-4bd0-8477-a1a49e40346b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:25.963709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6f793-afe3-4bd0-8477-a1a49e40346b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:25.963984 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6f793-afe3-4bd0-8477-a1a49e40346b None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:25.964717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6f793-afe3-4bd0-8477-a1a49e40346b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:25.965025 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6f793-afe3-4bd0-8477-a1a49e40346b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:25.966577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6f793-afe3-4bd0-8477-a1a49e40346b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:25.983316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 499/989] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:25.985995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:25.990524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6f793-afe3-4bd0-8477-a1a49e40346b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:26.037333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:26.067222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb0411-9dc9-462c-a4ed-e0f0f8eeabf8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:26.101770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6f793-afe3-4bd0-8477-a1a49e40346b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:26.106504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0e8b7-0390-44a4-a586-acb679a672cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:26.107201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0e8b7-0390-44a4-a586-acb679a672cb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:26.107483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0e8b7-0390-44a4-a586-acb679a672cb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:26.107758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0e8b7-0390-44a4-a586-acb679a672cb None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:26.108492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0e8b7-0390-44a4-a586-acb679a672cb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:26.108806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0e8b7-0390-44a4-a586-acb679a672cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:26.110280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0e8b7-0390-44a4-a586-acb679a672cb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:26.120580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 491/990] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:51:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:26.122576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0e8b7-0390-44a4-a586-acb679a672cb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:26.147520 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0889232-2a38-4327-a816-ad170a78cc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:26.148327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0889232-2a38-4327-a816-ad170a78cc94 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:26.148612 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0889232-2a38-4327-a816-ad170a78cc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:26.148887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0889232-2a38-4327-a816-ad170a78cc94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:26.156460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0889232-2a38-4327-a816-ad170a78cc94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:26.158095 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0889232-2a38-4327-a816-ad170a78cc94 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b0889232-2a38-4327-a816-ad170a78cc94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:26.172731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:26.173799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:26.181026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:26.181026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:26.181026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:26.181026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:26.181026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:26.181026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:26.181026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:26.181026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:26.181026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 500/991] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:26.181026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0889232-2a38-4327-a816-ad170a78cc94 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:26.233867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:26.243925 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0e8b7-0390-44a4-a586-acb679a672cb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:26.285421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0cbb75-bd46-4412-b50a-f528393203c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:26.286170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cbb75-bd46-4412-b50a-f528393203c4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:26.286400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cbb75-bd46-4412-b50a-f528393203c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:26.286673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cbb75-bd46-4412-b50a-f528393203c4 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:26.287389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0cbb75-bd46-4412-b50a-f528393203c4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:26.288514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0cbb75-bd46-4412-b50a-f528393203c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:26.289940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0cbb75-bd46-4412-b50a-f528393203c4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:26.295200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 492/992] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:51:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:26.296556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e0852-7b5d-4859-9a1d-587e428164a3 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:26.296846 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e0852-7b5d-4859-9a1d-587e428164a3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:26.297124 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e0852-7b5d-4859-9a1d-587e428164a3 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:26.298400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 493/993] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:26.301531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0889232-2a38-4327-a816-ad170a78cc94 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:26.303323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:26.303530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e0852-7b5d-4859-9a1d-587e428164a3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:26.341157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3551bdd-b50f-4113-977e-bb156c8af9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:26.341874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3551bdd-b50f-4113-977e-bb156c8af9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:26.342159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3551bdd-b50f-4113-977e-bb156c8af9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:26.342436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3551bdd-b50f-4113-977e-bb156c8af9b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:26.343236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3551bdd-b50f-4113-977e-bb156c8af9b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:26.343547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3551bdd-b50f-4113-977e-bb156c8af9b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:26.345014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3551bdd-b50f-4113-977e-bb156c8af9b9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:26.357711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 501/994] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:51:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:26.359021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593c557-9427-435e-9ff5-333d8bca7d27 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:26.359316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593c557-9427-435e-9ff5-333d8bca7d27 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:26.359596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593c557-9427-435e-9ff5-333d8bca7d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:26.378725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8593c557-9427-435e-9ff5-333d8bca7d27 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:26.380297 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8593c557-9427-435e-9ff5-333d8bca7d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:26.414682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e0852-7b5d-4859-9a1d-587e428164a3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:26.450556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad764b8-2aa1-4015-a265-8076574aed21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:26.451016 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 502/995] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:26.451195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad764b8-2aa1-4015-a265-8076574aed21 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:26.451438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad764b8-2aa1-4015-a265-8076574aed21 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:26.451626 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad764b8-2aa1-4015-a265-8076574aed21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:26.451879 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593c557-9427-435e-9ff5-333d8bca7d27 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:26.452541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad764b8-2aa1-4015-a265-8076574aed21 None admin] RBAC: Adding query filter params (interface=public, service_id=874a54a30e694bbb8e6f057a846e6cef, region_id=RegionOne) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:26.452851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad764b8-2aa1-4015-a265-8076574aed21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:26.454277 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad764b8-2aa1-4015-a265-8076574aed21 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:26.464640 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 494/996] 10.222.0.43 () {56 vars in 1251 bytes} [Mon Jul 4 07:51:26 2022] GET /identity/v3/endpoints?service_id=874a54a30e694bbb8e6f057a846e6cef&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:26.465874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad764b8-2aa1-4015-a265-8076574aed21 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:26.487133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:26.555203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:26.564958 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593c557-9427-435e-9ff5-333d8bca7d27 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:26.573045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad764b8-2aa1-4015-a265-8076574aed21 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:26.599512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40617cf0-fdab-4865-b6fb-945e49187cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:26.600296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40617cf0-fdab-4865-b6fb-945e49187cde None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:26.600579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40617cf0-fdab-4865-b6fb-945e49187cde None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:26.600856 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40617cf0-fdab-4865-b6fb-945e49187cde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:26.601712 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40617cf0-fdab-4865-b6fb-945e49187cde None admin] RBAC: Adding query filter params (role.id=41ba7c35817d4f368d33695911c9295e, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=80a45a063c0e4f1e9b2156fb2f4cd8b8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:26.602026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40617cf0-fdab-4865-b6fb-945e49187cde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:26.603578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40617cf0-fdab-4865-b6fb-945e49187cde None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:26.606200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1ef40-bc81-4874-a280-e9a93772dce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:26.606874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1ef40-bc81-4874-a280-e9a93772dce8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:26.607154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1ef40-bc81-4874-a280-e9a93772dce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:26.607427 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1ef40-bc81-4874-a280-e9a93772dce8 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:26.608104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e1ef40-bc81-4874-a280-e9a93772dce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:26.609458 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e1ef40-bc81-4874-a280-e9a93772dce8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:26.615818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 495/997] 10.222.0.43 () {56 vars in 1087 bytes} [Mon Jul 4 07:51:26 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 07:51:26.616713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1ef40-bc81-4874-a280-e9a93772dce8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:26.629743 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 503/998] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:51:26 2022] GET /identity/v3/role_assignments?user.id=80a45a063c0e4f1e9b2156fb2f4cd8b8&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=41ba7c35817d4f368d33695911c9295e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:26.728075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:26.752285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1ef40-bc81-4874-a280-e9a93772dce8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:26.795679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a6b94-1dfc-4705-965a-e18600b68ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:26.796536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a6b94-1dfc-4705-965a-e18600b68ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:26.796818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a6b94-1dfc-4705-965a-e18600b68ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:26.797095 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a6b94-1dfc-4705-965a-e18600b68ea6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:26.797931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a6b94-1dfc-4705-965a-e18600b68ea6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:26.799343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a6b94-1dfc-4705-965a-e18600b68ea6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7b2a6b94-1dfc-4705-965a-e18600b68ea6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:26.813235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:26.814570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:26.815533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 496/999] 10.222.0.43 () {56 vars in 1126 bytes} [Mon Jul 4 07:51:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:26.822894 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40617cf0-fdab-4865-b6fb-945e49187cde None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:26.934304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40617cf0-fdab-4865-b6fb-945e49187cde None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:26.971470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86945887-6236-4ffb-9c4b-8e7fa538bd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:26.972153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86945887-6236-4ffb-9c4b-8e7fa538bd89 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:26.972436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86945887-6236-4ffb-9c4b-8e7fa538bd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:26.972710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86945887-6236-4ffb-9c4b-8e7fa538bd89 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:26.973424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86945887-6236-4ffb-9c4b-8e7fa538bd89 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:26.973763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86945887-6236-4ffb-9c4b-8e7fa538bd89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:26.975496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86945887-6236-4ffb-9c4b-8e7fa538bd89 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:26.981018 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 504/1000] 10.222.0.43 () {56 vars in 1122 bytes} [Mon Jul 4 07:51:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:26.986636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a6b94-1dfc-4705-965a-e18600b68ea6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:27.092775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a6b94-1dfc-4705-965a-e18600b68ea6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:27.137358 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002a6dc-05f3-4f1e-af3c-59e2fb18a033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:27.138073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002a6dc-05f3-4f1e-af3c-59e2fb18a033 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:27.138361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002a6dc-05f3-4f1e-af3c-59e2fb18a033 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:27.138636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002a6dc-05f3-4f1e-af3c-59e2fb18a033 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:27.139348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002a6dc-05f3-4f1e-af3c-59e2fb18a033 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:27.139657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002a6dc-05f3-4f1e-af3c-59e2fb18a033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:27.141040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002a6dc-05f3-4f1e-af3c-59e2fb18a033 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:27.146571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 497/1001] 10.222.0.43 () {56 vars in 1122 bytes} [Mon Jul 4 07:51:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:27.152183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86945887-6236-4ffb-9c4b-8e7fa538bd89 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:27.175751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:27.263065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86945887-6236-4ffb-9c4b-8e7fa538bd89 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:27.302106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12ba54-138c-4f9c-a541-b728299fe061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:27.303271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12ba54-138c-4f9c-a541-b728299fe061 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:27.303828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12ba54-138c-4f9c-a541-b728299fe061 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:27.304308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12ba54-138c-4f9c-a541-b728299fe061 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:27.305263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12ba54-138c-4f9c-a541-b728299fe061 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:27.306969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12ba54-138c-4f9c-a541-b728299fe061 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:27.330316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 505/1002] 10.222.0.43 () {60 vars in 1144 bytes} [Mon Jul 4 07:51:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:28.689377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f089ed-a33b-4e49-bfe5-2470fca2748d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:28.690215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f089ed-a33b-4e49-bfe5-2470fca2748d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:28.690215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f089ed-a33b-4e49-bfe5-2470fca2748d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:28.691434 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 498/1003] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:28.697382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ecbf7-9de0-4707-bf77-07c6844bcc0f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:28.697983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ecbf7-9de0-4707-bf77-07c6844bcc0f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:28.697983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ecbf7-9de0-4707-bf77-07c6844bcc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:28.718235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-103ecbf7-9de0-4707-bf77-07c6844bcc0f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:28.720167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-103ecbf7-9de0-4707-bf77-07c6844bcc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:28.762718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93182d-958f-416a-92cc-d52ad0345a66 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:28.763003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93182d-958f-416a-92cc-d52ad0345a66 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:28.763288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93182d-958f-416a-92cc-d52ad0345a66 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:28.765033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 499/1004] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:28.770016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a865e-8776-408a-9b44-edd94c8a41dc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:28.770306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a865e-8776-408a-9b44-edd94c8a41dc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:28.770585 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a865e-8776-408a-9b44-edd94c8a41dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:28.790363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b16a865e-8776-408a-9b44-edd94c8a41dc None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:28.791590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b16a865e-8776-408a-9b44-edd94c8a41dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:28.794683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:28.813648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 506/1005] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:28 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:28.828961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ecbf7-9de0-4707-bf77-07c6844bcc0f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:28.845466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:28.870453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 500/1006] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:28 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:28.910220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a865e-8776-408a-9b44-edd94c8a41dc None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:28.957585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ecbf7-9de0-4707-bf77-07c6844bcc0f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:28.995947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac19f8-a91f-43dd-8641-a7f153f1b86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:28.996788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac19f8-a91f-43dd-8641-a7f153f1b86b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:28.997076 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac19f8-a91f-43dd-8641-a7f153f1b86b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:28.997346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac19f8-a91f-43dd-8641-a7f153f1b86b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:28.998044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac19f8-a91f-43dd-8641-a7f153f1b86b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:28.999549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac19f8-a91f-43dd-8641-a7f153f1b86b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-37ac19f8-a91f-43dd-8641-a7f153f1b86b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:29.013009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:29.015061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:29.016720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 507/1007] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:29.020543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac19f8-a91f-43dd-8641-a7f153f1b86b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:29.063871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a865e-8776-408a-9b44-edd94c8a41dc None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:29.101011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:29.101724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192259e1-303a-4d03-8069-4fe042590de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:29.102547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192259e1-303a-4d03-8069-4fe042590de5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:29.102852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192259e1-303a-4d03-8069-4fe042590de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:29.103093 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192259e1-303a-4d03-8069-4fe042590de5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:29.103758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192259e1-303a-4d03-8069-4fe042590de5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:29.105156 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192259e1-303a-4d03-8069-4fe042590de5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-192259e1-303a-4d03-8069-4fe042590de5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:29.121308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:29.122602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:29.123504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 501/1008] 10.222.0.43 () {56 vars in 1105 bytes} [Mon Jul 4 07:51:28 2022] GET /identity/v3/services/image => generated 86 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:29.130117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192259e1-303a-4d03-8069-4fe042590de5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:29.134334 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:29.150810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac19f8-a91f-43dd-8641-a7f153f1b86b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:29.193692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92bd59-0b00-4c73-8d9d-85f401a4147f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:29.194646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92bd59-0b00-4c73-8d9d-85f401a4147f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:29.194850 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92bd59-0b00-4c73-8d9d-85f401a4147f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:29.195123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92bd59-0b00-4c73-8d9d-85f401a4147f None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:29.195855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92bd59-0b00-4c73-8d9d-85f401a4147f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:29.196170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92bd59-0b00-4c73-8d9d-85f401a4147f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:29.197725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92bd59-0b00-4c73-8d9d-85f401a4147f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:29.204766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 508/1009] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:51:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:29.210869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92bd59-0b00-4c73-8d9d-85f401a4147f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:29.248775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192259e1-303a-4d03-8069-4fe042590de5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:29.289017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c45fa-fce6-40a7-a057-789719fbdeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:29.289734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c45fa-fce6-40a7-a057-789719fbdeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:29.290016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c45fa-fce6-40a7-a057-789719fbdeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:29.290288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c45fa-fce6-40a7-a057-789719fbdeb4 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:29.291005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c45fa-fce6-40a7-a057-789719fbdeb4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:29.291308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c45fa-fce6-40a7-a057-789719fbdeb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:29.292709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c45fa-fce6-40a7-a057-789719fbdeb4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:29.299581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 502/1010] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:29.305747 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c45fa-fce6-40a7-a057-789719fbdeb4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:29.330791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92bd59-0b00-4c73-8d9d-85f401a4147f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:29.368424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987aa4de-10d6-4bd8-8f49-e438cb8ba343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:29.369068 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987aa4de-10d6-4bd8-8f49-e438cb8ba343 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:29.369342 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987aa4de-10d6-4bd8-8f49-e438cb8ba343 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:29.369617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987aa4de-10d6-4bd8-8f49-e438cb8ba343 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:29.370371 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987aa4de-10d6-4bd8-8f49-e438cb8ba343 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:29.370675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987aa4de-10d6-4bd8-8f49-e438cb8ba343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:29.372052 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987aa4de-10d6-4bd8-8f49-e438cb8ba343 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:29.373555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:29.379335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 509/1011] 10.222.0.43 () {56 vars in 1112 bytes} [Mon Jul 4 07:51:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:29.385327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987aa4de-10d6-4bd8-8f49-e438cb8ba343 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:29.385885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:29.430826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c45fa-fce6-40a7-a057-789719fbdeb4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:29.471080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa102743-1983-455e-888e-85e1046d9606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:29.471761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa102743-1983-455e-888e-85e1046d9606 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:29.472012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa102743-1983-455e-888e-85e1046d9606 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:29.472309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa102743-1983-455e-888e-85e1046d9606 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:29.472988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa102743-1983-455e-888e-85e1046d9606 None admin] RBAC: Adding query filter params (type=image) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:29.473315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa102743-1983-455e-888e-85e1046d9606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:29.474710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa102743-1983-455e-888e-85e1046d9606 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:29.480017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 503/1012] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:29.481604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f68dce-d04d-47ef-9718-e95698b2b0c9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:29.481907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f68dce-d04d-47ef-9718-e95698b2b0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:29.482184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f68dce-d04d-47ef-9718-e95698b2b0c9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:29.483569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 504/1013] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:29.489499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6baf9d-d1e8-4e48-88c8-a402026c110e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:29.489889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6baf9d-d1e8-4e48-88c8-a402026c110e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:29.490098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6baf9d-d1e8-4e48-88c8-a402026c110e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:29.519062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ff6baf9d-d1e8-4e48-88c8-a402026c110e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:29.520169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ff6baf9d-d1e8-4e48-88c8-a402026c110e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:29.527215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987aa4de-10d6-4bd8-8f49-e438cb8ba343 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:29.577069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a08dde-a401-474e-9281-59a93b92dd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:29.578090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a08dde-a401-474e-9281-59a93b92dd89 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:29.578368 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a08dde-a401-474e-9281-59a93b92dd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:29.578652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a08dde-a401-474e-9281-59a93b92dd89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:29.579614 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a08dde-a401-474e-9281-59a93b92dd89 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:29.581661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a08dde-a401-474e-9281-59a93b92dd89 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:29.604676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 505/1014] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:29 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:29.606921 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 510/1015] 10.222.0.43 () {60 vars in 1144 bytes} [Mon Jul 4 07:51:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 07:51:29.633043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6baf9d-d1e8-4e48-88c8-a402026c110e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:29.721241 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:29.771119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6baf9d-d1e8-4e48-88c8-a402026c110e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:29.823473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2579f-e45f-4b19-9152-3c2a764e3433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:29.824316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2579f-e45f-4b19-9152-3c2a764e3433 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:29.824614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2579f-e45f-4b19-9152-3c2a764e3433 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:29.824879 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2579f-e45f-4b19-9152-3c2a764e3433 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:29.826181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2579f-e45f-4b19-9152-3c2a764e3433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:29.827723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2579f-e45f-4b19-9152-3c2a764e3433 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:29.829826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 506/1016] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:29.835553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a08dde-a401-474e-9281-59a93b92dd89 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:29.899665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:29.951883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a08dde-a401-474e-9281-59a93b92dd89 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:29.992293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba65d5-ac38-463d-b87b-a12b52f27e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:29.993360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba65d5-ac38-463d-b87b-a12b52f27e64 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:29.993913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba65d5-ac38-463d-b87b-a12b52f27e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:29.994393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba65d5-ac38-463d-b87b-a12b52f27e64 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:29.995344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba65d5-ac38-463d-b87b-a12b52f27e64 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:29.997023 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba65d5-ac38-463d-b87b-a12b52f27e64 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:30.029044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 511/1017] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:51:29 2022] POST /identity/v3/projects => generated 311 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:31.589979 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5400f4-7b45-44ab-9157-3aac20982d3b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:31.590597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5400f4-7b45-44ab-9157-3aac20982d3b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:31.590597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5400f4-7b45-44ab-9157-3aac20982d3b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:31.591893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 507/1018] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:31.597696 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16626cdf-e133-4bc0-9485-4973319fd93e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:31.598204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16626cdf-e133-4bc0-9485-4973319fd93e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:31.598298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16626cdf-e133-4bc0-9485-4973319fd93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:31.615940 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-16626cdf-e133-4bc0-9485-4973319fd93e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:31.617226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-16626cdf-e133-4bc0-9485-4973319fd93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:31.692702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 512/1019] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:31 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:31.715767 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5400f4-7b45-44ab-9157-3aac20982d3b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:31.825310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5400f4-7b45-44ab-9157-3aac20982d3b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:31.845707 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d53653-023e-40ce-8a2c-a3c3e931215a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:31.845886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d53653-023e-40ce-8a2c-a3c3e931215a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:31.846191 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d53653-023e-40ce-8a2c-a3c3e931215a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:31.847736 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 513/1020] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:31.854269 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a752ce3-d76b-47c6-910f-e838db443f6c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:31.854775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a752ce3-d76b-47c6-910f-e838db443f6c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:31.855261 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a752ce3-d76b-47c6-910f-e838db443f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:31.859499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:31.869806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474ba00-7a53-4823-9d61-6aab140190c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:31.870492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474ba00-7a53-4823-9d61-6aab140190c1 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:31.870776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474ba00-7a53-4823-9d61-6aab140190c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:31.871052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474ba00-7a53-4823-9d61-6aab140190c1 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:31.871801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474ba00-7a53-4823-9d61-6aab140190c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:31.873179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474ba00-7a53-4823-9d61-6aab140190c1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:31.886057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8a752ce3-d76b-47c6-910f-e838db443f6c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:31.888257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8a752ce3-d76b-47c6-910f-e838db443f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:31.889661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 508/1021] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:51:31 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:31.949589 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:32.043507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 514/1022] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:32.087007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474ba00-7a53-4823-9d61-6aab140190c1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:32.140041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1e247-e748-47d8-882c-98cfda3071f6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:32.140393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1e247-e748-47d8-882c-98cfda3071f6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:32.140769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1e247-e748-47d8-882c-98cfda3071f6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:32.142732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 515/1023] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:32.149986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09e802-e278-4b38-894f-aca48317ab76 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:32.150474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09e802-e278-4b38-894f-aca48317ab76 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:32.150969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09e802-e278-4b38-894f-aca48317ab76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:32.171624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fe09e802-e278-4b38-894f-aca48317ab76 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:32.172853 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fe09e802-e278-4b38-894f-aca48317ab76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:32.231529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474ba00-7a53-4823-9d61-6aab140190c1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:32.264964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 516/1024] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:32.291794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09e802-e278-4b38-894f-aca48317ab76 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:32.298187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e561441-714d-42e7-a1d2-6f4a1384517d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:32.299545 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e561441-714d-42e7-a1d2-6f4a1384517d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:32.300025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e561441-714d-42e7-a1d2-6f4a1384517d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:32.300383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e561441-714d-42e7-a1d2-6f4a1384517d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:32.317598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e561441-714d-42e7-a1d2-6f4a1384517d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1e561441-714d-42e7-a1d2-6f4a1384517d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:32.327933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:32.329164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:32.330183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:32.330183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:32.330183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:32.330183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:32.330183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:32.330183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:32.330183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:32.330183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:32.330183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:32.330183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:32.330183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 07:51:32.330183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:32.332812 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 509/1025] 10.222.0.43 () {56 vars in 1120 bytes} [Mon Jul 4 07:51:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:32.340099 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e561441-714d-42e7-a1d2-6f4a1384517d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:32.407843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:32.465437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:32.481274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e561441-714d-42e7-a1d2-6f4a1384517d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:32.481668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09e802-e278-4b38-894f-aca48317ab76 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:32.520414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9adee1-6b48-4469-beed-fde40d1c224d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:32.521145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9adee1-6b48-4469-beed-fde40d1c224d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:32.521428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9adee1-6b48-4469-beed-fde40d1c224d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:32.521747 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9adee1-6b48-4469-beed-fde40d1c224d None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:32.522521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9adee1-6b48-4469-beed-fde40d1c224d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:32.522838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9adee1-6b48-4469-beed-fde40d1c224d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:32.524289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9adee1-6b48-4469-beed-fde40d1c224d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:32.540488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 510/1026] 10.222.0.43 () {56 vars in 1115 bytes} [Mon Jul 4 07:51:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:32.542638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9adee1-6b48-4469-beed-fde40d1c224d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:32.556347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f315b-170a-45ae-8848-6b568cb62523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:32.557438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f315b-170a-45ae-8848-6b568cb62523 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:32.558130 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f315b-170a-45ae-8848-6b568cb62523 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:32.558991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f315b-170a-45ae-8848-6b568cb62523 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:32.561689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f315b-170a-45ae-8848-6b568cb62523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:32.563460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f315b-170a-45ae-8848-6b568cb62523 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:32.565657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 517/1027] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:32.572586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f315b-170a-45ae-8848-6b568cb62523 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:32.713900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9adee1-6b48-4469-beed-fde40d1c224d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:32.724157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f315b-170a-45ae-8848-6b568cb62523 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:32.730351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:32.767338 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50794463-093a-4778-82d4-e1287798b2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:32.768445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50794463-093a-4778-82d4-e1287798b2ae None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:32.768771 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50794463-093a-4778-82d4-e1287798b2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:32.769058 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50794463-093a-4778-82d4-e1287798b2ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:32.776263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f3abe-278c-440e-9cef-b703c621a27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:32.777044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f3abe-278c-440e-9cef-b703c621a27c None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:32.777404 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f3abe-278c-440e-9cef-b703c621a27c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:32.777658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f3abe-278c-440e-9cef-b703c621a27c None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:32.778425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f3abe-278c-440e-9cef-b703c621a27c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:32.779897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f3abe-278c-440e-9cef-b703c621a27c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:32.783801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50794463-093a-4778-82d4-e1287798b2ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:32.785897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50794463-093a-4778-82d4-e1287798b2ae None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:32.802427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 518/1028] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:51:32 2022] POST /identity/v3/users => generated 285 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-50794463-093a-4778-82d4-e1287798b2ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:32.811622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:32.813073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:32.814328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:32.814328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:32.814328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:32.814328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:32.814328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 511/1029] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:32.823444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f3abe-278c-440e-9cef-b703c621a27c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:32.853808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:32.978502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f3abe-278c-440e-9cef-b703c621a27c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:33.041526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e115ed5-7eea-4527-9835-ef56b3ba01dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:33.042437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e115ed5-7eea-4527-9835-ef56b3ba01dc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:33.043020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e115ed5-7eea-4527-9835-ef56b3ba01dc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:33.043301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e115ed5-7eea-4527-9835-ef56b3ba01dc None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:33.044229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e115ed5-7eea-4527-9835-ef56b3ba01dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:33.044543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e115ed5-7eea-4527-9835-ef56b3ba01dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:33.046615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e115ed5-7eea-4527-9835-ef56b3ba01dc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:33.061509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 519/1030] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:33.069852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50794463-093a-4778-82d4-e1287798b2ae None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:33.226183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50794463-093a-4778-82d4-e1287798b2ae None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:33.269220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5994c-3a1a-45c6-80de-700d676f3831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:33.270031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5994c-3a1a-45c6-80de-700d676f3831 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:33.270354 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5994c-3a1a-45c6-80de-700d676f3831 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:33.270593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5994c-3a1a-45c6-80de-700d676f3831 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:33.280749 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5994c-3a1a-45c6-80de-700d676f3831 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:33.282256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5994c-3a1a-45c6-80de-700d676f3831 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-4ae5994c-3a1a-45c6-80de-700d676f3831 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:51:33.297849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:33.299430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:33.300635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:33.300635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:33.300635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:51:33.300635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:51:33.300635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:51:33.300635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:51:33.300635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:51:33.300635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:51:33.300635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:51:33.300635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:33.300635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 512/1031] 10.222.0.43 () {56 vars in 1093 bytes} [Mon Jul 4 07:51:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:33.308817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e115ed5-7eea-4527-9835-ef56b3ba01dc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:33.423595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:33.462711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e115ed5-7eea-4527-9835-ef56b3ba01dc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:33.506989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5a269-920d-4b71-89ab-d9b81d3728fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:33.507941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5a269-920d-4b71-89ab-d9b81d3728fe None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:33.508197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5a269-920d-4b71-89ab-d9b81d3728fe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:33.508474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5a269-920d-4b71-89ab-d9b81d3728fe None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:33.509431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5a269-920d-4b71-89ab-d9b81d3728fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:33.509780 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5a269-920d-4b71-89ab-d9b81d3728fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:33.511398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5a269-920d-4b71-89ab-d9b81d3728fe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:33.538175 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 520/1032] 10.222.0.43 () {56 vars in 1133 bytes} [Mon Jul 4 07:51:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:33.540008 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5994c-3a1a-45c6-80de-700d676f3831 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:33.571858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:33.675995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5994c-3a1a-45c6-80de-700d676f3831 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:33.713651 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d9b2-b5e0-4c0f-b61c-5d899181da4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:33.714340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d9b2-b5e0-4c0f-b61c-5d899181da4d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:33.714995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d9b2-b5e0-4c0f-b61c-5d899181da4d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:33.714995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d9b2-b5e0-4c0f-b61c-5d899181da4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:33.804390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0d9b2-b5e0-4c0f-b61c-5d899181da4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:33.805725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0d9b2-b5e0-4c0f-b61c-5d899181da4d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:33.829918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:33.941669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 513/1033] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:51:33 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:51:33.946219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5a269-920d-4b71-89ab-d9b81d3728fe None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:34.077242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5a269-920d-4b71-89ab-d9b81d3728fe None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:34.134327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab861fb6-1cd6-411e-bdce-72f4c13ebd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:34.135223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab861fb6-1cd6-411e-bdce-72f4c13ebd60 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:34.135503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab861fb6-1cd6-411e-bdce-72f4c13ebd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:34.135801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab861fb6-1cd6-411e-bdce-72f4c13ebd60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:34.142923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:34.153667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab861fb6-1cd6-411e-bdce-72f4c13ebd60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:34.155134 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab861fb6-1cd6-411e-bdce-72f4c13ebd60 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ab861fb6-1cd6-411e-bdce-72f4c13ebd60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:34.174759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:34.182877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:34.184126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:34.184126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:34.184126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:34.184126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:34.184126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 521/1034] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:34.187478 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d9b2-b5e0-4c0f-b61c-5d899181da4d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:34.234369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caf844-8ef5-480c-9569-27bbf78b4115 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:34.234758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caf844-8ef5-480c-9569-27bbf78b4115 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:34.234930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caf844-8ef5-480c-9569-27bbf78b4115 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:34.236442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 522/1035] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:34.244401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835f639-9c0c-4a1f-b59e-337f0de71090 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:34.244974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835f639-9c0c-4a1f-b59e-337f0de71090 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:34.245401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835f639-9c0c-4a1f-b59e-337f0de71090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:34.269207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e835f639-9c0c-4a1f-b59e-337f0de71090 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:34.271540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e835f639-9c0c-4a1f-b59e-337f0de71090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:34.319995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d9b2-b5e0-4c0f-b61c-5d899181da4d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:34.342936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:34.358275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 523/1036] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:34.363933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525afb25-40e8-4a68-bbab-d3b9cc2408e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:34.364633 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525afb25-40e8-4a68-bbab-d3b9cc2408e0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:34.364917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525afb25-40e8-4a68-bbab-d3b9cc2408e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:34.365249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525afb25-40e8-4a68-bbab-d3b9cc2408e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:34.366516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525afb25-40e8-4a68-bbab-d3b9cc2408e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:34.366848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525afb25-40e8-4a68-bbab-d3b9cc2408e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:34.368384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525afb25-40e8-4a68-bbab-d3b9cc2408e0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:34.383229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 514/1037] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:34.389451 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525afb25-40e8-4a68-bbab-d3b9cc2408e0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:34.389957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835f639-9c0c-4a1f-b59e-337f0de71090 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:34.547942 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525afb25-40e8-4a68-bbab-d3b9cc2408e0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:34.564209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835f639-9c0c-4a1f-b59e-337f0de71090 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:34.590034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:34.605791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674df82-6cab-47bb-ab90-390808e1ecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:34.606731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674df82-6cab-47bb-ab90-390808e1ecd4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:34.607220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674df82-6cab-47bb-ab90-390808e1ecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:34.607608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674df82-6cab-47bb-ab90-390808e1ecd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:34.614053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18158a-0a99-4092-bee4-e16f915d65b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:34.615042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18158a-0a99-4092-bee4-e16f915d65b6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:34.615552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18158a-0a99-4092-bee4-e16f915d65b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:34.615958 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18158a-0a99-4092-bee4-e16f915d65b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:34.616790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18158a-0a99-4092-bee4-e16f915d65b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:34.618524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18158a-0a99-4092-bee4-e16f915d65b6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:34.619083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674df82-6cab-47bb-ab90-390808e1ecd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:34.620551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674df82-6cab-47bb-ab90-390808e1ecd4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0c18158a-0a99-4092-bee4-e16f915d65b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:34.636309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:34.637406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 07:51:34.643964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9674df82-6cab-47bb-ab90-390808e1ecd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:34.644752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:34.646461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:34.650756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:34.650756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:34.650756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:34.650756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:34.650756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:34.650756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:34.650756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:34.650756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:34.650756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 515/1038] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:34.652002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 524/1039] 10.222.0.43 () {56 vars in 1105 bytes} [Mon Jul 4 07:51:34 2022] GET /identity/v3/services/image => generated 86 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:34.652002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18158a-0a99-4092-bee4-e16f915d65b6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:34.652315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674df82-6cab-47bb-ab90-390808e1ecd4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:34.697443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:34.776104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674df82-6cab-47bb-ab90-390808e1ecd4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:34.784080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18158a-0a99-4092-bee4-e16f915d65b6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:34.837776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de82a-2006-48b5-bc2e-7b24fcfc81db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:34.838495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de82a-2006-48b5-bc2e-7b24fcfc81db None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:34.838759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de82a-2006-48b5-bc2e-7b24fcfc81db None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:34.839035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de82a-2006-48b5-bc2e-7b24fcfc81db None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:34.839851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608de82a-2006-48b5-bc2e-7b24fcfc81db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:34.840175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608de82a-2006-48b5-bc2e-7b24fcfc81db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:34.841541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffc733-5558-442f-9010-ddc19ca10945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:34.841701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608de82a-2006-48b5-bc2e-7b24fcfc81db None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:34.842659 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffc733-5558-442f-9010-ddc19ca10945 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:34.843092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffc733-5558-442f-9010-ddc19ca10945 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:34.843881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffc733-5558-442f-9010-ddc19ca10945 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:34.845523 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffc733-5558-442f-9010-ddc19ca10945 None admin] RBAC: Adding query filter params (name=image) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:34.846031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffc733-5558-442f-9010-ddc19ca10945 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:34.847596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffc733-5558-442f-9010-ddc19ca10945 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:34.854934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 525/1040] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:34.855528 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 516/1041] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:51:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:34.863064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffc733-5558-442f-9010-ddc19ca10945 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:34.863227 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de82a-2006-48b5-bc2e-7b24fcfc81db None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:34.935610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:34.984760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:34.988283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de82a-2006-48b5-bc2e-7b24fcfc81db None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:35.017041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffc733-5558-442f-9010-ddc19ca10945 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:35.027288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1679e-565d-4d59-af7b-1bd8ac002f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:35.027959 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1679e-565d-4d59-af7b-1bd8ac002f01 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:35.028258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1679e-565d-4d59-af7b-1bd8ac002f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:35.028507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1679e-565d-4d59-af7b-1bd8ac002f01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:35.029348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1679e-565d-4d59-af7b-1bd8ac002f01 None admin] RBAC: Adding query filter params (role.id=41ba7c35817d4f368d33695911c9295e, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=15211e42ec9d4c1699d877f0b55ce918) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:35.029682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1679e-565d-4d59-af7b-1bd8ac002f01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:35.031136 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1679e-565d-4d59-af7b-1bd8ac002f01 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:35.037087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 517/1042] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:51:34 2022] GET /identity/v3/role_assignments?user.id=15211e42ec9d4c1699d877f0b55ce918&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=41ba7c35817d4f368d33695911c9295e => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:35.038844 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b0690-7d4e-4468-be24-f82cca3c8fb4 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:35.039135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b0690-7d4e-4468-be24-f82cca3c8fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:35.039414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b0690-7d4e-4468-be24-f82cca3c8fb4 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:35.040744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 518/1043] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:35.047017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5c58c-c1ec-4e9e-b2ed-c6163ed82136 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:35.047285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5c58c-c1ec-4e9e-b2ed-c6163ed82136 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:35.047596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5c58c-c1ec-4e9e-b2ed-c6163ed82136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:35.066900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-97d5c58c-c1ec-4e9e-b2ed-c6163ed82136 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:35.068226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-97d5c58c-c1ec-4e9e-b2ed-c6163ed82136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:35.073615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4168ef2-12e0-4b5c-bd53-61f46323a7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:35.074399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4168ef2-12e0-4b5c-bd53-61f46323a7ec None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:35.074689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4168ef2-12e0-4b5c-bd53-61f46323a7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:35.074971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4168ef2-12e0-4b5c-bd53-61f46323a7ec None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:35.075744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4168ef2-12e0-4b5c-bd53-61f46323a7ec None admin] RBAC: Adding query filter params (type=image) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:35.076060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4168ef2-12e0-4b5c-bd53-61f46323a7ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:35.077580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4168ef2-12e0-4b5c-bd53-61f46323a7ec None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:35.089478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 526/1044] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:35.093804 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4168ef2-12e0-4b5c-bd53-61f46323a7ec None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:35.181110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 519/1045] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:35.205821 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5c58c-c1ec-4e9e-b2ed-c6163ed82136 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:35.257689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4168ef2-12e0-4b5c-bd53-61f46323a7ec None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:35.269273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:35.307726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236f9f5-4aaa-482d-b40a-f2a91f0e8c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:35.308429 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f9f5-4aaa-482d-b40a-f2a91f0e8c3d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:35.308966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f9f5-4aaa-482d-b40a-f2a91f0e8c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:35.309314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f9f5-4aaa-482d-b40a-f2a91f0e8c3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:35.310423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236f9f5-4aaa-482d-b40a-f2a91f0e8c3d None admin] RBAC: Adding query filter params (interface=public, service_id=697a891a2b004dd0873ef22411e86b04, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:35.310895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236f9f5-4aaa-482d-b40a-f2a91f0e8c3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:35.312540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236f9f5-4aaa-482d-b40a-f2a91f0e8c3d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:35.330977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 527/1046] 10.222.0.43 () {56 vars in 1251 bytes} [Mon Jul 4 07:51:35 2022] GET /identity/v3/endpoints?service_id=697a891a2b004dd0873ef22411e86b04&interface=public®ion_id=RegionOne => generated 194 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:35.332196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:35.336297 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236f9f5-4aaa-482d-b40a-f2a91f0e8c3d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:35.338371 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5c58c-c1ec-4e9e-b2ed-c6163ed82136 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:35.392401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9df45-d3ad-487a-980a-2fb9e2c0205e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:35.393685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9df45-d3ad-487a-980a-2fb9e2c0205e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:35.393989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9df45-d3ad-487a-980a-2fb9e2c0205e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:35.394311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9df45-d3ad-487a-980a-2fb9e2c0205e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:35.407425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9df45-d3ad-487a-980a-2fb9e2c0205e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-33b9df45-d3ad-487a-980a-2fb9e2c0205e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:35.416066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:35.417260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:35.419300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 520/1047] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:51:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:35.423592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9df45-d3ad-487a-980a-2fb9e2c0205e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:35.483375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236f9f5-4aaa-482d-b40a-f2a91f0e8c3d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:35.524132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4e323-36d1-4202-9881-69316e817fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:35.525092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4e323-36d1-4202-9881-69316e817fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:35.525501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4e323-36d1-4202-9881-69316e817fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:35.527075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4e323-36d1-4202-9881-69316e817fb9 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:35.527270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4e323-36d1-4202-9881-69316e817fb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:35.529178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4e323-36d1-4202-9881-69316e817fb9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:35.537800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 528/1048] 10.222.0.43 () {56 vars in 1087 bytes} [Mon Jul 4 07:51:35 2022] GET /identity/v3/services => generated 1957 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 07:51:35.566838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9df45-d3ad-487a-980a-2fb9e2c0205e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:35.611160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64450e-dfd9-4dd4-a6af-c88f0f441c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:35.611885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64450e-dfd9-4dd4-a6af-c88f0f441c27 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:35.612283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64450e-dfd9-4dd4-a6af-c88f0f441c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:35.612565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64450e-dfd9-4dd4-a6af-c88f0f441c27 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:35.613390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64450e-dfd9-4dd4-a6af-c88f0f441c27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:35.613717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64450e-dfd9-4dd4-a6af-c88f0f441c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:35.615216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64450e-dfd9-4dd4-a6af-c88f0f441c27 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:35.626105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 521/1049] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:51:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:35.632754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4e323-36d1-4202-9881-69316e817fb9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:35.675011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:35.759499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4e323-36d1-4202-9881-69316e817fb9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:35.802541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41626a8d-f9c5-4dd6-82a1-95b92a4f3275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:35.804252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41626a8d-f9c5-4dd6-82a1-95b92a4f3275 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:35.804534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41626a8d-f9c5-4dd6-82a1-95b92a4f3275 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:35.804822 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41626a8d-f9c5-4dd6-82a1-95b92a4f3275 None admin] PATH_INFO: `/v3/users/58dfef7f520f41078469c7843dde4b4d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:35.820902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41626a8d-f9c5-4dd6-82a1-95b92a4f3275 None admin] RBAC: Authorizing `identity:get_user(user_id=58dfef7f520f41078469c7843dde4b4d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:35.822428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41626a8d-f9c5-4dd6-82a1-95b92a4f3275 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:35.824408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 529/1050] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:51:35 2022] GET /identity/v3/users/58dfef7f520f41078469c7843dde4b4d => generated 285 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:35.830317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64450e-dfd9-4dd4-a6af-c88f0f441c27 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:35.851555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:35.951796 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64450e-dfd9-4dd4-a6af-c88f0f441c27 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:35.995210 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e02c90-bf9a-4eb5-b818-63a1c3f45497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:35.996129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e02c90-bf9a-4eb5-b818-63a1c3f45497 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:35.996571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e02c90-bf9a-4eb5-b818-63a1c3f45497 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:35.996990 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e02c90-bf9a-4eb5-b818-63a1c3f45497 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:36.072399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e02c90-bf9a-4eb5-b818-63a1c3f45497 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:36.074288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e02c90-bf9a-4eb5-b818-63a1c3f45497 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:36.095551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:36.209773 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 522/1051] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:51:35 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:51:36.223692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41626a8d-f9c5-4dd6-82a1-95b92a4f3275 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:36.340168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:36.348243 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41626a8d-f9c5-4dd6-82a1-95b92a4f3275 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:36.386312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859aef36-f50a-4176-acec-d2db430f44fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:36.387196 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859aef36-f50a-4176-acec-d2db430f44fa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:36.387479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859aef36-f50a-4176-acec-d2db430f44fa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:36.387753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859aef36-f50a-4176-acec-d2db430f44fa None admin] PATH_INFO: `/v3/projects/7ea5c46e310548c4bce4f87040a6eec9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:36.388958 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859aef36-f50a-4176-acec-d2db430f44fa None admin] RBAC: Authorizing `identity:get_project(project_id=7ea5c46e310548c4bce4f87040a6eec9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:36.390442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859aef36-f50a-4176-acec-d2db430f44fa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:36.392338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 530/1052] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:51:36 2022] GET /identity/v3/projects/7ea5c46e310548c4bce4f87040a6eec9 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:36.398584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e02c90-bf9a-4eb5-b818-63a1c3f45497 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:36.504938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e02c90-bf9a-4eb5-b818-63a1c3f45497 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:36.523215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:36.542149 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdfdad-57d5-4991-be22-f1597879d426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:36.542812 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdfdad-57d5-4991-be22-f1597879d426 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:36.543094 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdfdad-57d5-4991-be22-f1597879d426 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:36.543362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdfdad-57d5-4991-be22-f1597879d426 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:36.544192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdfdad-57d5-4991-be22-f1597879d426 None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=7ea5c46e310548c4bce4f87040a6eec9, user.id=58dfef7f520f41078469c7843dde4b4d) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:36.544632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdfdad-57d5-4991-be22-f1597879d426 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:36.546103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdfdad-57d5-4991-be22-f1597879d426 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:36.552351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 523/1053] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:51:36 2022] GET /identity/v3/role_assignments?user.id=58dfef7f520f41078469c7843dde4b4d&scope.project.id=7ea5c46e310548c4bce4f87040a6eec9&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:37.117701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c3abd-f3af-44af-8508-39fd8b98e23a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:37.117701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c3abd-f3af-44af-8508-39fd8b98e23a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:37.117701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c3abd-f3af-44af-8508-39fd8b98e23a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:37.120700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 531/1054] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:37.128867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc28504-1a29-41ee-bece-74df57c45edb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:37.129387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc28504-1a29-41ee-bece-74df57c45edb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:37.129387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc28504-1a29-41ee-bece-74df57c45edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:37.147548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8fc28504-1a29-41ee-bece-74df57c45edb None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:37.148880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8fc28504-1a29-41ee-bece-74df57c45edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:37.233486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 524/1055] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:37.257403 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c3abd-f3af-44af-8508-39fd8b98e23a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:37.368224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c3abd-f3af-44af-8508-39fd8b98e23a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:37.404658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31445997-6d4e-4d12-b953-fe9e8a146032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:37.405482 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31445997-6d4e-4d12-b953-fe9e8a146032 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:37.405783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31445997-6d4e-4d12-b953-fe9e8a146032 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:37.406042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31445997-6d4e-4d12-b953-fe9e8a146032 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:37.417412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31445997-6d4e-4d12-b953-fe9e8a146032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-31445997-6d4e-4d12-b953-fe9e8a146032 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:37.425836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:37.426982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:37.434230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 532/1056] 10.222.0.43 () {56 vars in 1120 bytes} [Mon Jul 4 07:51:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:37.434972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc28504-1a29-41ee-bece-74df57c45edb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:37.546556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc28504-1a29-41ee-bece-74df57c45edb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:37.557943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:37.583669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffeaa6-417f-4707-9983-c5e4cce26d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:37.584421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffeaa6-417f-4707-9983-c5e4cce26d72 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:37.584681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffeaa6-417f-4707-9983-c5e4cce26d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:37.584945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffeaa6-417f-4707-9983-c5e4cce26d72 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:37.585731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffeaa6-417f-4707-9983-c5e4cce26d72 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:37.586105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffeaa6-417f-4707-9983-c5e4cce26d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:37.587415 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffeaa6-417f-4707-9983-c5e4cce26d72 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:37.596645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 525/1057] 10.222.0.43 () {56 vars in 1115 bytes} [Mon Jul 4 07:51:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:37.602835 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31445997-6d4e-4d12-b953-fe9e8a146032 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:37.628723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3552d-d9b5-41e5-8332-8df8396761b1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:37.628944 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3552d-d9b5-41e5-8332-8df8396761b1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:37.629228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3552d-d9b5-41e5-8332-8df8396761b1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:37.631047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 526/1058] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:37.634925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:37.636894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241a4e1-5f9e-49a6-a81d-aaa859bace69 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:37.637152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241a4e1-5f9e-49a6-a81d-aaa859bace69 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:37.637438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241a4e1-5f9e-49a6-a81d-aaa859bace69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:37.655304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3241a4e1-5f9e-49a6-a81d-aaa859bace69 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:37.656530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3241a4e1-5f9e-49a6-a81d-aaa859bace69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:37.723991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31445997-6d4e-4d12-b953-fe9e8a146032 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:37.735629 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 527/1059] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:37.759334 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241a4e1-5f9e-49a6-a81d-aaa859bace69 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:37.767518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c829173-4a06-464e-8dc5-d9de6cf6e821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:37.768581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c829173-4a06-464e-8dc5-d9de6cf6e821 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:37.769051 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c829173-4a06-464e-8dc5-d9de6cf6e821 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:37.769492 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c829173-4a06-464e-8dc5-d9de6cf6e821 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:37.782287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c829173-4a06-464e-8dc5-d9de6cf6e821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:37.784002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c829173-4a06-464e-8dc5-d9de6cf6e821 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7c829173-4a06-464e-8dc5-d9de6cf6e821 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:37.803072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:37.804418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:37.805378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:37.805378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:37.805378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:37.805378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:37.805378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 533/1060] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:37.810114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c829173-4a06-464e-8dc5-d9de6cf6e821 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:37.870086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241a4e1-5f9e-49a6-a81d-aaa859bace69 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:37.871869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:37.898147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:37.911725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24018725-bce7-4c18-aaa3-f016f56c6396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:37.912967 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24018725-bce7-4c18-aaa3-f016f56c6396 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:37.913466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24018725-bce7-4c18-aaa3-f016f56c6396 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:37.913987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24018725-bce7-4c18-aaa3-f016f56c6396 None admin] PATH_INFO: `/v3/services/image` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:37.915110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24018725-bce7-4c18-aaa3-f016f56c6396 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:37.917443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24018725-bce7-4c18-aaa3-f016f56c6396 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:37.928255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c829173-4a06-464e-8dc5-d9de6cf6e821 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-24018725-bce7-4c18-aaa3-f016f56c6396 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:37.937390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:37.938724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 07:51:37.939656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:37.940457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 528/1061] 10.222.0.43 () {56 vars in 1105 bytes} [Mon Jul 4 07:51:37 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:37.948952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24018725-bce7-4c18-aaa3-f016f56c6396 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:37.991796 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cb0b2-a666-4697-a07f-3bc5f251d6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:37.993358 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cb0b2-a666-4697-a07f-3bc5f251d6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:37.993819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cb0b2-a666-4697-a07f-3bc5f251d6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:37.994620 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cb0b2-a666-4697-a07f-3bc5f251d6c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:37.996163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cb0b2-a666-4697-a07f-3bc5f251d6c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:37.996163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cb0b2-a666-4697-a07f-3bc5f251d6c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:37.997779 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cb0b2-a666-4697-a07f-3bc5f251d6c0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:38.018223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 534/1062] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:38.026308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cb0b2-a666-4697-a07f-3bc5f251d6c0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:38.091217 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24018725-bce7-4c18-aaa3-f016f56c6396 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:38.149363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d3866-61af-46aa-8944-32ca401d1e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:38.150045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d3866-61af-46aa-8944-32ca401d1e49 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:38.150845 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d3866-61af-46aa-8944-32ca401d1e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:38.151126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d3866-61af-46aa-8944-32ca401d1e49 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:38.152228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d3866-61af-46aa-8944-32ca401d1e49 None admin] RBAC: Adding query filter params (name=image) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:38.152543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d3866-61af-46aa-8944-32ca401d1e49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:38.154857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d3866-61af-46aa-8944-32ca401d1e49 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:38.162023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 529/1063] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:38.165329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cb0b2-a666-4697-a07f-3bc5f251d6c0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:38.169691 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d3866-61af-46aa-8944-32ca401d1e49 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:38.216675 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:38.220400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719e462-aecd-42af-a93c-ea61c7a03c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:38.221432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719e462-aecd-42af-a93c-ea61c7a03c89 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:38.221865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719e462-aecd-42af-a93c-ea61c7a03c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:38.222219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719e462-aecd-42af-a93c-ea61c7a03c89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:38.225978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:38.236155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c719e462-aecd-42af-a93c-ea61c7a03c89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:38.238432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c719e462-aecd-42af-a93c-ea61c7a03c89 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c719e462-aecd-42af-a93c-ea61c7a03c89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:51:38.257464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:38.258993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:38.259966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:38.259966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:38.259966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:51:38.259966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:51:38.259966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:51:38.259966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:51:38.259966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:51:38.259966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:51:38.259966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:51:38.259966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:38.259966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 535/1064] 10.222.0.43 () {56 vars in 1093 bytes} [Mon Jul 4 07:51:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:38.266632 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719e462-aecd-42af-a93c-ea61c7a03c89 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:38.326122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d3866-61af-46aa-8944-32ca401d1e49 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:38.380956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036b126-02f4-40c8-9f13-c09ee7b45684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:38.381893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036b126-02f4-40c8-9f13-c09ee7b45684 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:38.382291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036b126-02f4-40c8-9f13-c09ee7b45684 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:38.382651 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036b126-02f4-40c8-9f13-c09ee7b45684 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:38.383688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036b126-02f4-40c8-9f13-c09ee7b45684 None admin] RBAC: Adding query filter params (type=image) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:38.384231 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036b126-02f4-40c8-9f13-c09ee7b45684 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:38.386912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036b126-02f4-40c8-9f13-c09ee7b45684 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:38.389905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719e462-aecd-42af-a93c-ea61c7a03c89 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:38.398639 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 530/1065] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:38.400162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036b126-02f4-40c8-9f13-c09ee7b45684 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:38.440603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7038b-eb59-4cb8-901e-9d5f564dbc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:38.441546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7038b-eb59-4cb8-901e-9d5f564dbc41 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:38.441948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7038b-eb59-4cb8-901e-9d5f564dbc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:38.442362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7038b-eb59-4cb8-901e-9d5f564dbc41 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:38.443915 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7038b-eb59-4cb8-901e-9d5f564dbc41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:38.444284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7038b-eb59-4cb8-901e-9d5f564dbc41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:38.446739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7038b-eb59-4cb8-901e-9d5f564dbc41 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:38.476310 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 536/1066] 10.222.0.43 () {56 vars in 1133 bytes} [Mon Jul 4 07:51:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:38.481798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7038b-eb59-4cb8-901e-9d5f564dbc41 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:38.534206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:38.543518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036b126-02f4-40c8-9f13-c09ee7b45684 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:38.548387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:38.582820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7b690-53ab-415b-a596-ea8aa01b25de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:38.583518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7b690-53ab-415b-a596-ea8aa01b25de None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:38.583773 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7b690-53ab-415b-a596-ea8aa01b25de None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:38.584106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7b690-53ab-415b-a596-ea8aa01b25de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:38.585018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7b690-53ab-415b-a596-ea8aa01b25de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:38.586495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7b690-53ab-415b-a596-ea8aa01b25de None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:38.610115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 531/1067] 10.222.0.43 () {60 vars in 1144 bytes} [Mon Jul 4 07:51:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:38.611627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7038b-eb59-4cb8-901e-9d5f564dbc41 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:38.619809 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8fe8a-3ce5-4a6c-9458-8fcc9bffcfe1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:38.620155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8fe8a-3ce5-4a6c-9458-8fcc9bffcfe1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:38.620363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8fe8a-3ce5-4a6c-9458-8fcc9bffcfe1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:38.622181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 532/1068] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:38.628262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e33502-1161-4810-af9d-5954a7da8f8d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:38.628557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e33502-1161-4810-af9d-5954a7da8f8d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:38.628647 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e33502-1161-4810-af9d-5954a7da8f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:38.649149 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c8e33502-1161-4810-af9d-5954a7da8f8d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:38.650778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c8e33502-1161-4810-af9d-5954a7da8f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:38.673819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b00978-32b4-4c84-acb7-0aa7175e3358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:38.674851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b00978-32b4-4c84-acb7-0aa7175e3358 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:38.675178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b00978-32b4-4c84-acb7-0aa7175e3358 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:38.675479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b00978-32b4-4c84-acb7-0aa7175e3358 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:38.689733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b00978-32b4-4c84-acb7-0aa7175e3358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:38.691475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b00978-32b4-4c84-acb7-0aa7175e3358 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-19b00978-32b4-4c84-acb7-0aa7175e3358 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:38.712381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:38.721741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:38.723000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:38.723000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:38.723000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:38.723000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:38.723000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 537/1069] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:38.723000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b00978-32b4-4c84-acb7-0aa7175e3358 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:38.768902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 533/1070] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:38.791785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e33502-1161-4810-af9d-5954a7da8f8d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:38.873378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:38.876613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b00978-32b4-4c84-acb7-0aa7175e3358 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:38.893565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:38.920244 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70a770-92e2-4f2b-8788-72480ddc047b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:38.920824 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70a770-92e2-4f2b-8788-72480ddc047b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:38.921126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70a770-92e2-4f2b-8788-72480ddc047b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:38.921463 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70a770-92e2-4f2b-8788-72480ddc047b None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:38.922140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70a770-92e2-4f2b-8788-72480ddc047b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:38.922512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70a770-92e2-4f2b-8788-72480ddc047b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:38.923847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70a770-92e2-4f2b-8788-72480ddc047b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:38.924167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e33502-1161-4810-af9d-5954a7da8f8d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:38.938040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 538/1071] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:38.946218 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70a770-92e2-4f2b-8788-72480ddc047b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:38.971957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30cbdd-24dd-4f0c-8838-9bab8f2b163f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:38.973361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30cbdd-24dd-4f0c-8838-9bab8f2b163f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:38.973819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30cbdd-24dd-4f0c-8838-9bab8f2b163f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:38.974243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30cbdd-24dd-4f0c-8838-9bab8f2b163f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:38.988226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30cbdd-24dd-4f0c-8838-9bab8f2b163f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9c30cbdd-24dd-4f0c-8838-9bab8f2b163f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:38.996604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:38.997701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:38.998660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 534/1072] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:51:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:39.006811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30cbdd-24dd-4f0c-8838-9bab8f2b163f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:39.071070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70a770-92e2-4f2b-8788-72480ddc047b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:39.122315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22dbe6-6b3a-4ff7-860f-b353a117a2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:39.123324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22dbe6-6b3a-4ff7-860f-b353a117a2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:39.123830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22dbe6-6b3a-4ff7-860f-b353a117a2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:39.124326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22dbe6-6b3a-4ff7-860f-b353a117a2e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:39.132923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30cbdd-24dd-4f0c-8838-9bab8f2b163f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:39.137508 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22dbe6-6b3a-4ff7-860f-b353a117a2e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:39.139034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22dbe6-6b3a-4ff7-860f-b353a117a2e4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1a22dbe6-6b3a-4ff7-860f-b353a117a2e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:39.157689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:39.158732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:39.159717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:39.159717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:39.159717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:39.159717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:39.159717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:39.159717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:39.159717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:39.159717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:39.159717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 539/1073] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:39.165277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22dbe6-6b3a-4ff7-860f-b353a117a2e4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:39.193953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:39.197168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862b226-a277-4a1d-badf-a50ffc921f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:39.198050 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862b226-a277-4a1d-badf-a50ffc921f7a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:39.198373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862b226-a277-4a1d-badf-a50ffc921f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:39.198762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862b226-a277-4a1d-badf-a50ffc921f7a None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:39.199736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862b226-a277-4a1d-badf-a50ffc921f7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:39.200123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862b226-a277-4a1d-badf-a50ffc921f7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:39.202060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862b226-a277-4a1d-badf-a50ffc921f7a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:39.220311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 535/1074] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:51:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:39.225122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862b226-a277-4a1d-badf-a50ffc921f7a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:39.229955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:39.302200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22dbe6-6b3a-4ff7-860f-b353a117a2e4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:39.354181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed2116-b45e-4515-be15-92e672b380a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:39.355158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed2116-b45e-4515-be15-92e672b380a1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:39.355325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862b226-a277-4a1d-badf-a50ffc921f7a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:39.355475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed2116-b45e-4515-be15-92e672b380a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:39.355791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed2116-b45e-4515-be15-92e672b380a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:39.356657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed2116-b45e-4515-be15-92e672b380a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:39.357211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed2116-b45e-4515-be15-92e672b380a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:39.359623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed2116-b45e-4515-be15-92e672b380a1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:39.385672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 540/1075] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:51:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:39.391238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed2116-b45e-4515-be15-92e672b380a1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:39.401169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec0676-850c-42f0-86cf-917966f08d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:39.402118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec0676-850c-42f0-86cf-917966f08d35 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:39.402399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec0676-850c-42f0-86cf-917966f08d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:39.402676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec0676-850c-42f0-86cf-917966f08d35 None admin] PATH_INFO: `/v3/users/58dfef7f520f41078469c7843dde4b4d` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:39.404039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec0676-850c-42f0-86cf-917966f08d35 None admin] RBAC: Authorizing `identity:get_user(user_id=58dfef7f520f41078469c7843dde4b4d)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:39.405517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec0676-850c-42f0-86cf-917966f08d35 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:39.413292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 536/1076] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:51:39 2022] GET /identity/v3/users/58dfef7f520f41078469c7843dde4b4d => generated 285 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:39.415540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec0676-850c-42f0-86cf-917966f08d35 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:39.505289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:39.506629 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:39.529985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed2116-b45e-4515-be15-92e672b380a1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:39.550946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec0676-850c-42f0-86cf-917966f08d35 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:39.570672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fdd47-7f3f-41d2-857d-fc9a5f7656ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:39.571586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fdd47-7f3f-41d2-857d-fc9a5f7656ec None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:39.571914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fdd47-7f3f-41d2-857d-fc9a5f7656ec None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:39.572232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fdd47-7f3f-41d2-857d-fc9a5f7656ec None admin] PATH_INFO: `/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/15211e42ec9d4c1699d877f0b55ce918/roles/41ba7c35817d4f368d33695911c9295e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:39.575531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fdd47-7f3f-41d2-857d-fc9a5f7656ec None admin] RBAC: Authorizing `identity:create_grant(project_id=1f2d85c9e3d14bdabfb63fe505d83714, user_id=15211e42ec9d4c1699d877f0b55ce918, role_id=41ba7c35817d4f368d33695911c9295e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:39.577000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fdd47-7f3f-41d2-857d-fc9a5f7656ec None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:39.592733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 541/1077] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:51:39 2022] PUT /identity/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/15211e42ec9d4c1699d877f0b55ce918/roles/41ba7c35817d4f368d33695911c9295e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 07:51:39.593138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7a99e-bbd8-4557-affe-f363e7b34227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:39.594029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a99e-bbd8-4557-affe-f363e7b34227 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:39.594319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a99e-bbd8-4557-affe-f363e7b34227 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:39.594590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a99e-bbd8-4557-affe-f363e7b34227 None admin] PATH_INFO: `/v3/projects/7ea5c46e310548c4bce4f87040a6eec9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:39.595897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7a99e-bbd8-4557-affe-f363e7b34227 None admin] RBAC: Authorizing `identity:get_project(project_id=7ea5c46e310548c4bce4f87040a6eec9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:39.597405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7a99e-bbd8-4557-affe-f363e7b34227 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:39.599600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 537/1078] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:51:39 2022] GET /identity/v3/projects/7ea5c46e310548c4bce4f87040a6eec9 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:39.605893 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fdd47-7f3f-41d2-857d-fc9a5f7656ec None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:39.750116 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fdd47-7f3f-41d2-857d-fc9a5f7656ec None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:39.794229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24626226-4a68-4204-b0d3-e3ce2a2a1ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:39.795080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24626226-4a68-4204-b0d3-e3ce2a2a1ea7 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:39.795360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24626226-4a68-4204-b0d3-e3ce2a2a1ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:39.795637 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24626226-4a68-4204-b0d3-e3ce2a2a1ea7 None admin] PATH_INFO: `/v3/projects/7ea5c46e310548c4bce4f87040a6eec9/users/58dfef7f520f41078469c7843dde4b4d/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:39.797985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24626226-4a68-4204-b0d3-e3ce2a2a1ea7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ea5c46e310548c4bce4f87040a6eec9, user_id=58dfef7f520f41078469c7843dde4b4d, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:39.799644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24626226-4a68-4204-b0d3-e3ce2a2a1ea7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:39.810669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:39.817759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 542/1079] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:51:39 2022] PUT /identity/v3/projects/7ea5c46e310548c4bce4f87040a6eec9/users/58dfef7f520f41078469c7843dde4b4d/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 07:51:40.838900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7414d-46ca-44ac-9062-0f31e54f109a None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:40.839540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7414d-46ca-44ac-9062-0f31e54f109a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:40.839540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7414d-46ca-44ac-9062-0f31e54f109a None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:40.840853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 538/1080] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:40.846945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15b165-df42-4524-8c14-eb7a93699309 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:40.847549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15b165-df42-4524-8c14-eb7a93699309 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:40.847549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15b165-df42-4524-8c14-eb7a93699309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:40.866341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6c15b165-df42-4524-8c14-eb7a93699309 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:40.867610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6c15b165-df42-4524-8c14-eb7a93699309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:40.949903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 543/1081] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:40.972437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7414d-46ca-44ac-9062-0f31e54f109a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:41.095844 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7414d-46ca-44ac-9062-0f31e54f109a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:41.137594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cece30-2946-4599-a5bf-2b384c3278ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:41.138294 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cece30-2946-4599-a5bf-2b384c3278ab None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:41.138580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cece30-2946-4599-a5bf-2b384c3278ab None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:41.138858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cece30-2946-4599-a5bf-2b384c3278ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:41.149071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:41.217408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cece30-2946-4599-a5bf-2b384c3278ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:41.218927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cece30-2946-4599-a5bf-2b384c3278ab None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:41.324149 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 539/1082] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:51:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:51:41.330153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15b165-df42-4524-8c14-eb7a93699309 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:41.435846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15b165-df42-4524-8c14-eb7a93699309 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:41.465676 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:41.472578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c6fb0-648f-4330-b014-1fd713879b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:41.473468 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c6fb0-648f-4330-b014-1fd713879b1f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:41.473787 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c6fb0-648f-4330-b014-1fd713879b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:41.474071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c6fb0-648f-4330-b014-1fd713879b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:41.485246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c6fb0-648f-4330-b014-1fd713879b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:41.486777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c6fb0-648f-4330-b014-1fd713879b1f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b62c6fb0-648f-4330-b014-1fd713879b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:41.505800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:41.507192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:41.508136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:41.508136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:41.508136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:41.508136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:41.509780 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 544/1083] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:41.513702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cece30-2946-4599-a5bf-2b384c3278ab None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:41.556105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:41.621327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cece30-2946-4599-a5bf-2b384c3278ab None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:41.660774 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76160763-7acb-48fe-bc18-2936213a0146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:41.662393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76160763-7acb-48fe-bc18-2936213a0146 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:41.662393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76160763-7acb-48fe-bc18-2936213a0146 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:41.662716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76160763-7acb-48fe-bc18-2936213a0146 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:41.663668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76160763-7acb-48fe-bc18-2936213a0146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:41.663973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76160763-7acb-48fe-bc18-2936213a0146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:41.666021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76160763-7acb-48fe-bc18-2936213a0146 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:41.679792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 540/1084] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:41.713971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c5dda-1581-44e4-8d8d-949daf43f121 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:41.714154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c5dda-1581-44e4-8d8d-949daf43f121 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:41.714439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c5dda-1581-44e4-8d8d-949daf43f121 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:41.715948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 545/1085] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:41.722171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c258e0f-a94c-4a51-9189-0059c295416f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:41.722513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c258e0f-a94c-4a51-9189-0059c295416f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:41.722745 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c258e0f-a94c-4a51-9189-0059c295416f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:41.742507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8c258e0f-a94c-4a51-9189-0059c295416f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:41.743783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8c258e0f-a94c-4a51-9189-0059c295416f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:41.827265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 541/1086] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:41.851615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c5dda-1581-44e4-8d8d-949daf43f121 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:41.973950 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c5dda-1581-44e4-8d8d-949daf43f121 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:42.014352 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b36ed1-902b-43ae-bca8-9bd3d98554ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:42.015191 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b36ed1-902b-43ae-bca8-9bd3d98554ed None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:42.015475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b36ed1-902b-43ae-bca8-9bd3d98554ed None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:42.015755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b36ed1-902b-43ae-bca8-9bd3d98554ed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:42.019400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc7238-de65-47cc-9ddc-0c7f98352def None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:42.019876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc7238-de65-47cc-9ddc-0c7f98352def None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:42.019976 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc7238-de65-47cc-9ddc-0c7f98352def None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:42.021405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 542/1087] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:42.026872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a3908-74eb-45d6-a9eb-dde7a710168d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:42.027161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a3908-74eb-45d6-a9eb-dde7a710168d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:42.027438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a3908-74eb-45d6-a9eb-dde7a710168d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:42.032193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b36ed1-902b-43ae-bca8-9bd3d98554ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-19b36ed1-902b-43ae-bca8-9bd3d98554ed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:42.041100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:42.042389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:42.043281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 546/1088] 10.222.0.43 () {56 vars in 1120 bytes} [Mon Jul 4 07:51:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:42.044549 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-841a3908-74eb-45d6-a9eb-dde7a710168d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:42.045795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-841a3908-74eb-45d6-a9eb-dde7a710168d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:42.048551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b36ed1-902b-43ae-bca8-9bd3d98554ed None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:42.070311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:42.095060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:42.117458 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 543/1089] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 07:51:42.141517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a3908-74eb-45d6-a9eb-dde7a710168d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:42.160087 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b36ed1-902b-43ae-bca8-9bd3d98554ed None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:42.195785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fda6e6-75b0-4b7c-8d18-d88b463aa999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:42.196474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fda6e6-75b0-4b7c-8d18-d88b463aa999 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:42.196759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fda6e6-75b0-4b7c-8d18-d88b463aa999 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:42.197033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fda6e6-75b0-4b7c-8d18-d88b463aa999 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:42.197815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fda6e6-75b0-4b7c-8d18-d88b463aa999 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:42.198116 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fda6e6-75b0-4b7c-8d18-d88b463aa999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:42.199506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fda6e6-75b0-4b7c-8d18-d88b463aa999 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:42.207983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 547/1090] 10.222.0.43 () {56 vars in 1115 bytes} [Mon Jul 4 07:51:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:42.214645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fda6e6-75b0-4b7c-8d18-d88b463aa999 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:42.244746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a3908-74eb-45d6-a9eb-dde7a710168d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:42.283290 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1a66e-9045-4e6d-bea4-72a449298021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:42.284160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1a66e-9045-4e6d-bea4-72a449298021 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:42.284438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1a66e-9045-4e6d-bea4-72a449298021 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:42.284757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1a66e-9045-4e6d-bea4-72a449298021 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:42.295325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1a66e-9045-4e6d-bea4-72a449298021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-02c1a66e-9045-4e6d-bea4-72a449298021 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:51:42.303179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:42.304473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:51:42.305357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 544/1091] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:51:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:42.310401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1a66e-9045-4e6d-bea4-72a449298021 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:42.319326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fda6e6-75b0-4b7c-8d18-d88b463aa999 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:42.322137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:42.334504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:42.355698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866fc36-fc1f-4281-a63c-23871a476237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:42.356510 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866fc36-fc1f-4281-a63c-23871a476237 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:42.356787 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866fc36-fc1f-4281-a63c-23871a476237 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:42.357058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866fc36-fc1f-4281-a63c-23871a476237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:42.368308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866fc36-fc1f-4281-a63c-23871a476237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:42.369864 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866fc36-fc1f-4281-a63c-23871a476237 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6866fc36-fc1f-4281-a63c-23871a476237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:42.387633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:42.388658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:42.389765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:42.389765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:42.389765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:42.389765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:42.389765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 548/1092] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:42.394685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866fc36-fc1f-4281-a63c-23871a476237 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:42.416552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1a66e-9045-4e6d-bea4-72a449298021 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:42.449125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210f10c-394a-4b6a-9f04-4a7f11cdfa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:42.449839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210f10c-394a-4b6a-9f04-4a7f11cdfa75 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:42.450130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210f10c-394a-4b6a-9f04-4a7f11cdfa75 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:42.450409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210f10c-394a-4b6a-9f04-4a7f11cdfa75 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:42.451191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210f10c-394a-4b6a-9f04-4a7f11cdfa75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:42.451499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210f10c-394a-4b6a-9f04-4a7f11cdfa75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:42.452886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210f10c-394a-4b6a-9f04-4a7f11cdfa75 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:42.466040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 545/1093] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:51:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:42.467089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210f10c-394a-4b6a-9f04-4a7f11cdfa75 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:42.497548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866fc36-fc1f-4281-a63c-23871a476237 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:42.532664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761879-a4b3-4255-b399-6c2960ba8d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:42.533337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761879-a4b3-4255-b399-6c2960ba8d3f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:42.533652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761879-a4b3-4255-b399-6c2960ba8d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:42.533925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761879-a4b3-4255-b399-6c2960ba8d3f None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:42.534651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36761879-a4b3-4255-b399-6c2960ba8d3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:42.534954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36761879-a4b3-4255-b399-6c2960ba8d3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:42.536335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36761879-a4b3-4255-b399-6c2960ba8d3f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:42.548216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 549/1094] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:42.551176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:42.554463 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761879-a4b3-4255-b399-6c2960ba8d3f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:42.571973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210f10c-394a-4b6a-9f04-4a7f11cdfa75 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:42.572594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:42.606425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a39f30-9e85-4fce-84fa-f5ae3305957e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:42.607314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a39f30-9e85-4fce-84fa-f5ae3305957e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:42.607598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a39f30-9e85-4fce-84fa-f5ae3305957e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:42.607876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a39f30-9e85-4fce-84fa-f5ae3305957e None admin] PATH_INFO: `/v3/users/58dfef7f520f41078469c7843dde4b4d` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:42.609101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a39f30-9e85-4fce-84fa-f5ae3305957e None admin] RBAC: Authorizing `identity:get_user(user_id=58dfef7f520f41078469c7843dde4b4d)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:42.610578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a39f30-9e85-4fce-84fa-f5ae3305957e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:42.612269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 546/1095] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:51:42 2022] GET /identity/v3/users/58dfef7f520f41078469c7843dde4b4d => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:42.618665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a39f30-9e85-4fce-84fa-f5ae3305957e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:42.661880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761879-a4b3-4255-b399-6c2960ba8d3f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:42.696997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37aa08-2541-4a3a-ad53-2cc41d353a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:42.697839 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37aa08-2541-4a3a-ad53-2cc41d353a2a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:42.698112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37aa08-2541-4a3a-ad53-2cc41d353a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:42.698395 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37aa08-2541-4a3a-ad53-2cc41d353a2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:42.706445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37aa08-2541-4a3a-ad53-2cc41d353a2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:42.707949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37aa08-2541-4a3a-ad53-2cc41d353a2a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5e37aa08-2541-4a3a-ad53-2cc41d353a2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:51:42.723697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:42.724983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:42.726034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:42.726034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:42.726034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:51:42.726034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:51:42.726034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:51:42.726034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:51:42.726034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:51:42.726034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:51:42.726034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 07:51:42.726034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:42.726034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 550/1096] 10.222.0.43 () {56 vars in 1093 bytes} [Mon Jul 4 07:51:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:51:42.731557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37aa08-2541-4a3a-ad53-2cc41d353a2a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:42.733161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a39f30-9e85-4fce-84fa-f5ae3305957e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:42.772629 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468abb27-e32a-4b03-8af2-2d0699da0c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:42.773214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468abb27-e32a-4b03-8af2-2d0699da0c62 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:42.773499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468abb27-e32a-4b03-8af2-2d0699da0c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:42.773797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468abb27-e32a-4b03-8af2-2d0699da0c62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:42.793059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:42.815563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:42.840172 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37aa08-2541-4a3a-ad53-2cc41d353a2a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:42.845753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468abb27-e32a-4b03-8af2-2d0699da0c62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:42.847259 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468abb27-e32a-4b03-8af2-2d0699da0c62 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:42.875992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956927ca-6365-419b-9411-cbe06ab2c60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:42.876683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956927ca-6365-419b-9411-cbe06ab2c60f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:42.876962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956927ca-6365-419b-9411-cbe06ab2c60f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:42.877238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956927ca-6365-419b-9411-cbe06ab2c60f None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:42.878136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956927ca-6365-419b-9411-cbe06ab2c60f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:42.878451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956927ca-6365-419b-9411-cbe06ab2c60f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:42.880252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956927ca-6365-419b-9411-cbe06ab2c60f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:42.899896 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 551/1097] 10.222.0.43 () {56 vars in 1133 bytes} [Mon Jul 4 07:51:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:42.905987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956927ca-6365-419b-9411-cbe06ab2c60f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:42.949261 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 547/1098] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:51:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 07:51:42.957741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468abb27-e32a-4b03-8af2-2d0699da0c62 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:43.014398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:43.021573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956927ca-6365-419b-9411-cbe06ab2c60f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:43.058743 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9218a19-cc26-433f-ba56-56167c3ad7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:43.059663 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9218a19-cc26-433f-ba56-56167c3ad7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:43.060043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9218a19-cc26-433f-ba56-56167c3ad7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:43.060224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9218a19-cc26-433f-ba56-56167c3ad7a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:43.063113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:43.066570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468abb27-e32a-4b03-8af2-2d0699da0c62 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:43.102303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f40cc-93ab-41d9-9102-1a13fe374e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:43.103168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f40cc-93ab-41d9-9102-1a13fe374e11 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:43.103452 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f40cc-93ab-41d9-9102-1a13fe374e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:43.103733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f40cc-93ab-41d9-9102-1a13fe374e11 None admin] PATH_INFO: `/v3/projects/7ea5c46e310548c4bce4f87040a6eec9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:43.104992 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f40cc-93ab-41d9-9102-1a13fe374e11 None admin] RBAC: Authorizing `identity:get_project(project_id=7ea5c46e310548c4bce4f87040a6eec9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:43.106466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f40cc-93ab-41d9-9102-1a13fe374e11 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:43.113169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 548/1099] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:51:42 2022] GET /identity/v3/projects/7ea5c46e310548c4bce4f87040a6eec9 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:43.114223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f40cc-93ab-41d9-9102-1a13fe374e11 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:43.136811 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9218a19-cc26-433f-ba56-56167c3ad7a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:43.138320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9218a19-cc26-433f-ba56-56167c3ad7a5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:43.230999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f40cc-93ab-41d9-9102-1a13fe374e11 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:43.248524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:43.258326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 552/1100] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:51:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 07:51:43.264220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9218a19-cc26-433f-ba56-56167c3ad7a5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:43.271301 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97c88-65ea-410b-88e3-5b1ed1adba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:43.272175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97c88-65ea-410b-88e3-5b1ed1adba96 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:43.272707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97c88-65ea-410b-88e3-5b1ed1adba96 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:43.273049 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97c88-65ea-410b-88e3-5b1ed1adba96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:43.273926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97c88-65ea-410b-88e3-5b1ed1adba96 None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=7ea5c46e310548c4bce4f87040a6eec9, user.id=58dfef7f520f41078469c7843dde4b4d) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:43.274245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97c88-65ea-410b-88e3-5b1ed1adba96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:43.275704 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97c88-65ea-410b-88e3-5b1ed1adba96 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:43.281710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 549/1101] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:51:43 2022] GET /identity/v3/role_assignments?user.id=58dfef7f520f41078469c7843dde4b4d&scope.project.id=7ea5c46e310548c4bce4f87040a6eec9&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:43.299449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:43.378788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9218a19-cc26-433f-ba56-56167c3ad7a5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:43.421652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294cc90-55e2-4ffb-8888-0887de57362f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:43.422487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294cc90-55e2-4ffb-8888-0887de57362f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:43.422765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294cc90-55e2-4ffb-8888-0887de57362f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:43.423039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294cc90-55e2-4ffb-8888-0887de57362f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:43.435661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294cc90-55e2-4ffb-8888-0887de57362f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:43.437171 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294cc90-55e2-4ffb-8888-0887de57362f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7294cc90-55e2-4ffb-8888-0887de57362f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:51:43.457397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:43.458545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:43.459563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:51:43.459563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:51:43.459563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:51:43.459563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:43.459563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 553/1102] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:43.465185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97c88-65ea-410b-88e3-5b1ed1adba96 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:43.577920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97c88-65ea-410b-88e3-5b1ed1adba96 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:43.617338 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0da659-9d86-41a0-bb86-72fad52a739c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:43.618045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0da659-9d86-41a0-bb86-72fad52a739c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:43.618323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0da659-9d86-41a0-bb86-72fad52a739c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:43.618597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0da659-9d86-41a0-bb86-72fad52a739c None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:43.619328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0da659-9d86-41a0-bb86-72fad52a739c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:43.619633 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0da659-9d86-41a0-bb86-72fad52a739c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:43.621028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0da659-9d86-41a0-bb86-72fad52a739c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:43.635624 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 550/1103] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:43.641346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294cc90-55e2-4ffb-8888-0887de57362f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:43.734043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:43.751734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294cc90-55e2-4ffb-8888-0887de57362f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:43.787677 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b2c76-4c78-41a9-b0f1-62167e6e745c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:43.788690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b2c76-4c78-41a9-b0f1-62167e6e745c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:43.788975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b2c76-4c78-41a9-b0f1-62167e6e745c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:43.789247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b2c76-4c78-41a9-b0f1-62167e6e745c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:43.796712 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b2c76-4c78-41a9-b0f1-62167e6e745c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:43.798231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b2c76-4c78-41a9-b0f1-62167e6e745c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7a9b2c76-4c78-41a9-b0f1-62167e6e745c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:51:43.813005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:51:43.814195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:51:43.815155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:51:43.815155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:51:43.815155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:51:43.815155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:51:43.815155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:51:43.815155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:51:43.815155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:51:43.815155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:51:43.815155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 554/1104] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:51:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:51:43.820693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0da659-9d86-41a0-bb86-72fad52a739c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:43.865152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:43.934957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0da659-9d86-41a0-bb86-72fad52a739c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:43.983634 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf1857-a02c-40bf-9c83-dd113f84e97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:43.984319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf1857-a02c-40bf-9c83-dd113f84e97e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:43.984610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf1857-a02c-40bf-9c83-dd113f84e97e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:43.984888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf1857-a02c-40bf-9c83-dd113f84e97e None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:43.985721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf1857-a02c-40bf-9c83-dd113f84e97e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:43.986036 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf1857-a02c-40bf-9c83-dd113f84e97e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:43.987499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf1857-a02c-40bf-9c83-dd113f84e97e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:44.000881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 551/1105] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:51:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:44.008252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b2c76-4c78-41a9-b0f1-62167e6e745c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:44.137131 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b2c76-4c78-41a9-b0f1-62167e6e745c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:44.171221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3294a-d25a-4b3a-abbc-3767e4e47bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:44.171912 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3294a-d25a-4b3a-abbc-3767e4e47bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:44.172215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3294a-d25a-4b3a-abbc-3767e4e47bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:44.172478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3294a-d25a-4b3a-abbc-3767e4e47bf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:44.173320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3294a-d25a-4b3a-abbc-3767e4e47bf4 None admin] RBAC: Adding query filter params (role.id=41ba7c35817d4f368d33695911c9295e, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=15211e42ec9d4c1699d877f0b55ce918) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:44.173774 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3294a-d25a-4b3a-abbc-3767e4e47bf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:44.175252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3294a-d25a-4b3a-abbc-3767e4e47bf4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:44.182223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 555/1106] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:51:44 2022] GET /identity/v3/role_assignments?user.id=15211e42ec9d4c1699d877f0b55ce918&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=41ba7c35817d4f368d33695911c9295e => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:45.373488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5db35-2788-4b68-a8bc-8cfc1317e575 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:45.374184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5db35-2788-4b68-a8bc-8cfc1317e575 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:45.374184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5db35-2788-4b68-a8bc-8cfc1317e575 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:45.375362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 552/1107] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:45.381009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea002a8-7fb1-4e7a-961d-cf248af6c775 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:45.381443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea002a8-7fb1-4e7a-961d-cf248af6c775 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:45.381545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea002a8-7fb1-4e7a-961d-cf248af6c775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:45.399401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1ea002a8-7fb1-4e7a-961d-cf248af6c775 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:45.400624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1ea002a8-7fb1-4e7a-961d-cf248af6c775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:45.403551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:45.479905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 556/1108] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:45.505079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5db35-2788-4b68-a8bc-8cfc1317e575 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:45.604041 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:45.608503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5db35-2788-4b68-a8bc-8cfc1317e575 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:45.644167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55739be3-e9ea-4996-a296-c9fe312aa6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:45.644881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55739be3-e9ea-4996-a296-c9fe312aa6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:45.645162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55739be3-e9ea-4996-a296-c9fe312aa6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:45.645433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55739be3-e9ea-4996-a296-c9fe312aa6e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:45.646716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55739be3-e9ea-4996-a296-c9fe312aa6e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:45.648159 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55739be3-e9ea-4996-a296-c9fe312aa6e5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:45.649799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 553/1109] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:45.655860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea002a8-7fb1-4e7a-961d-cf248af6c775 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:45.760488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea002a8-7fb1-4e7a-961d-cf248af6c775 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:45.795761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a70f46-6943-453a-8fce-2cbed83b93c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:45.796433 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a70f46-6943-453a-8fce-2cbed83b93c9 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:45.796719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a70f46-6943-453a-8fce-2cbed83b93c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:45.796997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a70f46-6943-453a-8fce-2cbed83b93c9 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:45.797750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a70f46-6943-453a-8fce-2cbed83b93c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:45.799205 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a70f46-6943-453a-8fce-2cbed83b93c9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:45.809474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:45.822447 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 557/1110] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:51:45 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:47.885778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57e324-0a03-4827-aa48-eef067b571ee None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:47.886379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57e324-0a03-4827-aa48-eef067b571ee None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:47.886379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57e324-0a03-4827-aa48-eef067b571ee None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:47.887711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 554/1111] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:47.893046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f330084-64fc-4d4c-8a62-8b89ae6d73e9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:47.893472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f330084-64fc-4d4c-8a62-8b89ae6d73e9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:47.893583 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f330084-64fc-4d4c-8a62-8b89ae6d73e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:47.912530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6f330084-64fc-4d4c-8a62-8b89ae6d73e9 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:47.913975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6f330084-64fc-4d4c-8a62-8b89ae6d73e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:47.985857 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 558/1112] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:48.010481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57e324-0a03-4827-aa48-eef067b571ee None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:48.115927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57e324-0a03-4827-aa48-eef067b571ee None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:48.151825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205ad5a-166a-49cf-b641-a30180de8dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:48.152604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205ad5a-166a-49cf-b641-a30180de8dcb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:48.152882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205ad5a-166a-49cf-b641-a30180de8dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:48.153154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205ad5a-166a-49cf-b641-a30180de8dcb None admin] PATH_INFO: `/v3/roles/794ff5f5032d40a98e67fb0df718b75c` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:48.154646 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205ad5a-166a-49cf-b641-a30180de8dcb None admin] RBAC: Authorizing `identity:get_role(role_id=794ff5f5032d40a98e67fb0df718b75c)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:48.156020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205ad5a-166a-49cf-b641-a30180de8dcb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:48.157428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 555/1113] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:51:48 2022] GET /identity/v3/roles/794ff5f5032d40a98e67fb0df718b75c => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:48.163580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f330084-64fc-4d4c-8a62-8b89ae6d73e9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:48.270947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f330084-64fc-4d4c-8a62-8b89ae6d73e9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:48.301379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:48.311810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013ada0-eaf5-4bdd-aafd-66f2d0b162ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:48.312735 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013ada0-eaf5-4bdd-aafd-66f2d0b162ce None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:48.313015 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013ada0-eaf5-4bdd-aafd-66f2d0b162ce None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:48.313287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013ada0-eaf5-4bdd-aafd-66f2d0b162ce None admin] PATH_INFO: `/v3/users/3ff977d4518f4e4798180ebde83442a7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:48.327816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013ada0-eaf5-4bdd-aafd-66f2d0b162ce None admin] RBAC: Authorizing `identity:get_user(user_id=3ff977d4518f4e4798180ebde83442a7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:48.329293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013ada0-eaf5-4bdd-aafd-66f2d0b162ce None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:48.331054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 559/1114] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:51:48 2022] GET /identity/v3/users/3ff977d4518f4e4798180ebde83442a7 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:48.337311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205ad5a-166a-49cf-b641-a30180de8dcb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:48.375388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:48.444460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205ad5a-166a-49cf-b641-a30180de8dcb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:48.480104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23ff3c-dced-44ec-9a0f-1ff121b12ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:48.480755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23ff3c-dced-44ec-9a0f-1ff121b12ccd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:48.481039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23ff3c-dced-44ec-9a0f-1ff121b12ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:48.481315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23ff3c-dced-44ec-9a0f-1ff121b12ccd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:48.553053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23ff3c-dced-44ec-9a0f-1ff121b12ccd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:48.554656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23ff3c-dced-44ec-9a0f-1ff121b12ccd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:48.589814 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:48.656760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 556/1115] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:51:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:51:48.662977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013ada0-eaf5-4bdd-aafd-66f2d0b162ce None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:48.768276 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013ada0-eaf5-4bdd-aafd-66f2d0b162ce None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:48.805181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2aabe-8250-4cfb-a683-46af231992af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:48.806090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2aabe-8250-4cfb-a683-46af231992af None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:48.806374 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2aabe-8250-4cfb-a683-46af231992af None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:48.806653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2aabe-8250-4cfb-a683-46af231992af None admin] PATH_INFO: `/v3/projects/7ea5c46e310548c4bce4f87040a6eec9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:48.808004 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2aabe-8250-4cfb-a683-46af231992af None admin] RBAC: Authorizing `identity:get_project(project_id=7ea5c46e310548c4bce4f87040a6eec9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:48.809486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2aabe-8250-4cfb-a683-46af231992af None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:48.811270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 560/1116] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:51:48 2022] GET /identity/v3/projects/7ea5c46e310548c4bce4f87040a6eec9 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:48.818010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23ff3c-dced-44ec-9a0f-1ff121b12ccd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:48.921657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23ff3c-dced-44ec-9a0f-1ff121b12ccd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:48.956381 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b17477-826b-4360-b50e-830a3eb117a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:48.957017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b17477-826b-4360-b50e-830a3eb117a8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:48.957297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b17477-826b-4360-b50e-830a3eb117a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:48.957557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b17477-826b-4360-b50e-830a3eb117a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:48.958411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b17477-826b-4360-b50e-830a3eb117a8 None admin] RBAC: Adding query filter params (role.id=794ff5f5032d40a98e67fb0df718b75c, scope.project.id=7ea5c46e310548c4bce4f87040a6eec9, user.id=3ff977d4518f4e4798180ebde83442a7) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:48.958716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b17477-826b-4360-b50e-830a3eb117a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:48.960284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b17477-826b-4360-b50e-830a3eb117a8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:48.963176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:48.966361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 557/1117] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:51:48 2022] GET /identity/v3/role_assignments?user.id=3ff977d4518f4e4798180ebde83442a7&scope.project.id=7ea5c46e310548c4bce4f87040a6eec9&role.id=794ff5f5032d40a98e67fb0df718b75c => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:50.979728 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37add5dc-97d4-46e1-9f2b-aaaf382b0b62 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:50.980332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37add5dc-97d4-46e1-9f2b-aaaf382b0b62 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:50.980332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37add5dc-97d4-46e1-9f2b-aaaf382b0b62 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:50.981666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 561/1118] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:50.987472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87409a97-b289-45ff-8c6b-edc5e231e017 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:50.987905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87409a97-b289-45ff-8c6b-edc5e231e017 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:50.988016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87409a97-b289-45ff-8c6b-edc5e231e017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:51.006000 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-87409a97-b289-45ff-8c6b-edc5e231e017 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:51.007290 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-87409a97-b289-45ff-8c6b-edc5e231e017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:51.077904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 558/1119] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:51.102355 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37add5dc-97d4-46e1-9f2b-aaaf382b0b62 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:51.162669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:51.214049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37add5dc-97d4-46e1-9f2b-aaaf382b0b62 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:51.249959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d9e06-9058-4c84-b2df-c12f99b3d4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:51.250735 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d9e06-9058-4c84-b2df-c12f99b3d4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:51.251016 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d9e06-9058-4c84-b2df-c12f99b3d4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:51.251284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d9e06-9058-4c84-b2df-c12f99b3d4b7 None admin] PATH_INFO: `/v3/roles/794ff5f5032d40a98e67fb0df718b75c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:51.252799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d9e06-9058-4c84-b2df-c12f99b3d4b7 None admin] RBAC: Authorizing `identity:get_role(role_id=794ff5f5032d40a98e67fb0df718b75c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:51.254210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d9e06-9058-4c84-b2df-c12f99b3d4b7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:51.255987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 562/1120] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:51:51 2022] GET /identity/v3/roles/794ff5f5032d40a98e67fb0df718b75c => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:51.262381 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87409a97-b289-45ff-8c6b-edc5e231e017 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:51.362779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87409a97-b289-45ff-8c6b-edc5e231e017 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:51.391671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:51.398232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1ee14-e93e-4b4d-8271-01b03fa92274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:51.399105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1ee14-e93e-4b4d-8271-01b03fa92274 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:51.399378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1ee14-e93e-4b4d-8271-01b03fa92274 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:51.399650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1ee14-e93e-4b4d-8271-01b03fa92274 None admin] PATH_INFO: `/v3/users/3ff977d4518f4e4798180ebde83442a7` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:51.400845 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1ee14-e93e-4b4d-8271-01b03fa92274 None admin] RBAC: Authorizing `identity:get_user(user_id=3ff977d4518f4e4798180ebde83442a7)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:51.402330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1ee14-e93e-4b4d-8271-01b03fa92274 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:51.403970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 559/1121] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:51:51 2022] GET /identity/v3/users/3ff977d4518f4e4798180ebde83442a7 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:51:51.410313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d9e06-9058-4c84-b2df-c12f99b3d4b7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:51.516980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d9e06-9058-4c84-b2df-c12f99b3d4b7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:51.538147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:51.554541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57ebb8-c78a-4b81-9e6d-c19158a67462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:51.555411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57ebb8-c78a-4b81-9e6d-c19158a67462 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:51.555693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57ebb8-c78a-4b81-9e6d-c19158a67462 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:51.555970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57ebb8-c78a-4b81-9e6d-c19158a67462 None admin] PATH_INFO: `/v3/projects/7ea5c46e310548c4bce4f87040a6eec9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:51.557173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57ebb8-c78a-4b81-9e6d-c19158a67462 None admin] RBAC: Authorizing `identity:get_project(project_id=7ea5c46e310548c4bce4f87040a6eec9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:51.558666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57ebb8-c78a-4b81-9e6d-c19158a67462 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:51.560448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 563/1122] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:51:51 2022] GET /identity/v3/projects/7ea5c46e310548c4bce4f87040a6eec9 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:51.566801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1ee14-e93e-4b4d-8271-01b03fa92274 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:51.672502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1ee14-e93e-4b4d-8271-01b03fa92274 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:51.709519 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718eb9e8-2d99-4871-9117-f99589f0800a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:51.710459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718eb9e8-2d99-4871-9117-f99589f0800a None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:51.710743 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718eb9e8-2d99-4871-9117-f99589f0800a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:51.711022 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718eb9e8-2d99-4871-9117-f99589f0800a None admin] PATH_INFO: `/v3/projects/7ea5c46e310548c4bce4f87040a6eec9/users/3ff977d4518f4e4798180ebde83442a7/roles/794ff5f5032d40a98e67fb0df718b75c` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:51.713291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718eb9e8-2d99-4871-9117-f99589f0800a None admin] RBAC: Authorizing `identity:create_grant(project_id=7ea5c46e310548c4bce4f87040a6eec9, user_id=3ff977d4518f4e4798180ebde83442a7, role_id=794ff5f5032d40a98e67fb0df718b75c)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:51.714783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718eb9e8-2d99-4871-9117-f99589f0800a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:51.729918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 560/1123] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:51:51 2022] PUT /identity/v3/projects/7ea5c46e310548c4bce4f87040a6eec9/users/3ff977d4518f4e4798180ebde83442a7/roles/794ff5f5032d40a98e67fb0df718b75c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:51:53.748557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14746c4-5b9f-4b06-89f0-ad95ce4990d4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:53.749125 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14746c4-5b9f-4b06-89f0-ad95ce4990d4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:53.749125 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14746c4-5b9f-4b06-89f0-ad95ce4990d4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:53.750541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 564/1124] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:53.756449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be44f1-52f9-42eb-959c-e9a90092a1ac None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:53.756905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be44f1-52f9-42eb-959c-e9a90092a1ac None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:53.757101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be44f1-52f9-42eb-959c-e9a90092a1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:53.775600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e9be44f1-52f9-42eb-959c-e9a90092a1ac None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:53.776825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e9be44f1-52f9-42eb-959c-e9a90092a1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:53.836140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:53.852477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 561/1125] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:53.877477 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14746c4-5b9f-4b06-89f0-ad95ce4990d4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:53.988968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14746c4-5b9f-4b06-89f0-ad95ce4990d4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:54.029098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291ee53-a9a5-4b33-845d-e5ec64bd42b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:54.029890 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291ee53-a9a5-4b33-845d-e5ec64bd42b1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:54.030172 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291ee53-a9a5-4b33-845d-e5ec64bd42b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:54.030448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291ee53-a9a5-4b33-845d-e5ec64bd42b1 None admin] PATH_INFO: `/v3/roles/794ff5f5032d40a98e67fb0df718b75c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:54.031996 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291ee53-a9a5-4b33-845d-e5ec64bd42b1 None admin] RBAC: Authorizing `identity:get_role(role_id=794ff5f5032d40a98e67fb0df718b75c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:54.033382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291ee53-a9a5-4b33-845d-e5ec64bd42b1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:54.034837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 565/1126] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:51:53 2022] GET /identity/v3/roles/794ff5f5032d40a98e67fb0df718b75c => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:54.041102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be44f1-52f9-42eb-959c-e9a90092a1ac None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:54.146010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be44f1-52f9-42eb-959c-e9a90092a1ac None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:54.181108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58e62b-481c-4707-aebc-c36850bfa8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:54.182026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58e62b-481c-4707-aebc-c36850bfa8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:54.182304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58e62b-481c-4707-aebc-c36850bfa8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:54.182574 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58e62b-481c-4707-aebc-c36850bfa8a4 None admin] PATH_INFO: `/v3/users/3ff977d4518f4e4798180ebde83442a7` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:54.183816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58e62b-481c-4707-aebc-c36850bfa8a4 None admin] RBAC: Authorizing `identity:get_user(user_id=3ff977d4518f4e4798180ebde83442a7)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:54.185273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58e62b-481c-4707-aebc-c36850bfa8a4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:54.186983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 562/1127] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:51:54 2022] GET /identity/v3/users/3ff977d4518f4e4798180ebde83442a7 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:54.193112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291ee53-a9a5-4b33-845d-e5ec64bd42b1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:54.251153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:54.304161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291ee53-a9a5-4b33-845d-e5ec64bd42b1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:54.340017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5461c9f-a2f5-40e4-b64c-7aa7b1678872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:54.340674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5461c9f-a2f5-40e4-b64c-7aa7b1678872 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:54.340952 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5461c9f-a2f5-40e4-b64c-7aa7b1678872 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:54.341228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5461c9f-a2f5-40e4-b64c-7aa7b1678872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:54.412283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5461c9f-a2f5-40e4-b64c-7aa7b1678872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:54.414064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5461c9f-a2f5-40e4-b64c-7aa7b1678872 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:54.470375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:54.518602 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 566/1128] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:51:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:51:54.525811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58e62b-481c-4707-aebc-c36850bfa8a4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:54.586072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:54.632056 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58e62b-481c-4707-aebc-c36850bfa8a4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:54.667342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061a1ee-99e9-4188-b78d-8d9fa57deee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:54.668207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061a1ee-99e9-4188-b78d-8d9fa57deee0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:54.668488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061a1ee-99e9-4188-b78d-8d9fa57deee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:54.668760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061a1ee-99e9-4188-b78d-8d9fa57deee0 None admin] PATH_INFO: `/v3/projects/7ea5c46e310548c4bce4f87040a6eec9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:54.669987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061a1ee-99e9-4188-b78d-8d9fa57deee0 None admin] RBAC: Authorizing `identity:get_project(project_id=7ea5c46e310548c4bce4f87040a6eec9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:54.671467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061a1ee-99e9-4188-b78d-8d9fa57deee0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:54.673195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 563/1129] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:51:54 2022] GET /identity/v3/projects/7ea5c46e310548c4bce4f87040a6eec9 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:54.680087 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5461c9f-a2f5-40e4-b64c-7aa7b1678872 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:54.783327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5461c9f-a2f5-40e4-b64c-7aa7b1678872 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:54.820905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15659d0-9d76-430a-ab5f-5cd6e962d30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:54.821554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15659d0-9d76-430a-ab5f-5cd6e962d30d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:54.821873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15659d0-9d76-430a-ab5f-5cd6e962d30d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:54.822151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15659d0-9d76-430a-ab5f-5cd6e962d30d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:54.822998 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15659d0-9d76-430a-ab5f-5cd6e962d30d None admin] RBAC: Adding query filter params (role.id=794ff5f5032d40a98e67fb0df718b75c, scope.project.id=7ea5c46e310548c4bce4f87040a6eec9, user.id=3ff977d4518f4e4798180ebde83442a7) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:51:54.823324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15659d0-9d76-430a-ab5f-5cd6e962d30d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:54.824771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15659d0-9d76-430a-ab5f-5cd6e962d30d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:54.831098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 567/1130] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:51:54 2022] GET /identity/v3/role_assignments?user.id=3ff977d4518f4e4798180ebde83442a7&scope.project.id=7ea5c46e310548c4bce4f87040a6eec9&role.id=794ff5f5032d40a98e67fb0df718b75c => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:56.871558 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b957f-f854-4a4a-acd0-f86a810ba198 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:56.872129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b957f-f854-4a4a-acd0-f86a810ba198 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:56.872129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b957f-f854-4a4a-acd0-f86a810ba198 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:56.873461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 564/1131] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:51:56.878839 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015e3b2-fe2b-487b-a7eb-6ab9a4680f1f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:56.879265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015e3b2-fe2b-487b-a7eb-6ab9a4680f1f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:56.879356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015e3b2-fe2b-487b-a7eb-6ab9a4680f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:56.898172 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d015e3b2-fe2b-487b-a7eb-6ab9a4680f1f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:56.899415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d015e3b2-fe2b-487b-a7eb-6ab9a4680f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:56.915181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:56.972614 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 568/1132] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:56.997155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b957f-f854-4a4a-acd0-f86a810ba198 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:57.103953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b957f-f854-4a4a-acd0-f86a810ba198 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:57.129055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:57.140437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cb59d-de60-493d-bcda-0ad8b0880f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:57.141238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cb59d-de60-493d-bcda-0ad8b0880f34 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:57.141526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cb59d-de60-493d-bcda-0ad8b0880f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:57.141833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cb59d-de60-493d-bcda-0ad8b0880f34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:57.143029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cb59d-de60-493d-bcda-0ad8b0880f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:57.144472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cb59d-de60-493d-bcda-0ad8b0880f34 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:57.146164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 565/1133] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:57.152359 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015e3b2-fe2b-487b-a7eb-6ab9a4680f1f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:57.258737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015e3b2-fe2b-487b-a7eb-6ab9a4680f1f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:57.295532 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3483b-f325-416e-ab4b-20b2864afc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:57.296206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3483b-f325-416e-ab4b-20b2864afc5a None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:57.296485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3483b-f325-416e-ab4b-20b2864afc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:57.296758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3483b-f325-416e-ab4b-20b2864afc5a None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:57.297464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3483b-f325-416e-ab4b-20b2864afc5a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:57.298944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3483b-f325-416e-ab4b-20b2864afc5a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:57.327479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 569/1134] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:51:57 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:51:59.385503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdd547-16da-42e3-95fb-580685a0e098 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:59.386103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdd547-16da-42e3-95fb-580685a0e098 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:59.386103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdd547-16da-42e3-95fb-580685a0e098 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:59.387406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 566/1135] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:51:59.392847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b30a84-92cb-4708-a99e-aee022b45530 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:59.393330 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b30a84-92cb-4708-a99e-aee022b45530 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:59.393432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b30a84-92cb-4708-a99e-aee022b45530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:59.399289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:59.413926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b7b30a84-92cb-4708-a99e-aee022b45530 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:51:59.415149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b7b30a84-92cb-4708-a99e-aee022b45530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:51:59.486558 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 570/1136] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:51:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:51:59.512758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fdd547-16da-42e3-95fb-580685a0e098 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:59.618240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fdd547-16da-42e3-95fb-580685a0e098 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:59.666033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac5593-3804-4dcf-b5d7-db8d20f2f2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:59.666749 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac5593-3804-4dcf-b5d7-db8d20f2f2dc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:59.667040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac5593-3804-4dcf-b5d7-db8d20f2f2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:59.667316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac5593-3804-4dcf-b5d7-db8d20f2f2dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:59.668588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac5593-3804-4dcf-b5d7-db8d20f2f2dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:59.670097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac5593-3804-4dcf-b5d7-db8d20f2f2dc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:59.671754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 567/1137] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:51:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:51:59.678463 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b30a84-92cb-4708-a99e-aee022b45530 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:51:59.788031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b30a84-92cb-4708-a99e-aee022b45530 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:51:59.819028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:51:59.825386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd2c4a-edd1-474f-8193-3b41e01b9ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:51:59.826127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd2c4a-edd1-474f-8193-3b41e01b9ca9 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:51:59.826415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd2c4a-edd1-474f-8193-3b41e01b9ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:51:59.826689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd2c4a-edd1-474f-8193-3b41e01b9ca9 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:51:59.827413 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd2c4a-edd1-474f-8193-3b41e01b9ca9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:51:59.828899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd2c4a-edd1-474f-8193-3b41e01b9ca9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:51:59.853325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 571/1138] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:51:59 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:52:01.940997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b23d4-8668-4544-85ed-dc859fb2cc1c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:01.941603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b23d4-8668-4544-85ed-dc859fb2cc1c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:01.941603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b23d4-8668-4544-85ed-dc859fb2cc1c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:01.942961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 568/1139] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:52:01.948533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d1655-a9d2-43c6-8edf-6d95e7a9a17c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:01.948989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d1655-a9d2-43c6-8edf-6d95e7a9a17c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:01.949096 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d1655-a9d2-43c6-8edf-6d95e7a9a17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:01.966912 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3e2d1655-a9d2-43c6-8edf-6d95e7a9a17c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:52:01.968113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3e2d1655-a9d2-43c6-8edf-6d95e7a9a17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:52:02.040563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 572/1140] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:52:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:52:02.065248 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b23d4-8668-4544-85ed-dc859fb2cc1c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:02.132943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:02.176948 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b23d4-8668-4544-85ed-dc859fb2cc1c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:02.214173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dab60-9489-4fad-8010-eeb2814fcb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:02.215016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dab60-9489-4fad-8010-eeb2814fcb9e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:02.215337 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dab60-9489-4fad-8010-eeb2814fcb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:02.215597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dab60-9489-4fad-8010-eeb2814fcb9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:02.227075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dab60-9489-4fad-8010-eeb2814fcb9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2c9dab60-9489-4fad-8010-eeb2814fcb9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:52:02.234721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:52:02.235863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:52:02.237170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 569/1141] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:52:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:52:02.242704 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d1655-a9d2-43c6-8edf-6d95e7a9a17c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:02.346471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d1655-a9d2-43c6-8edf-6d95e7a9a17c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:02.364815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:02.384799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61789083-855d-4ff4-9a3f-be933a02d32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:02.385482 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61789083-855d-4ff4-9a3f-be933a02d32f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:02.385814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61789083-855d-4ff4-9a3f-be933a02d32f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:02.386113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61789083-855d-4ff4-9a3f-be933a02d32f None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:02.386882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61789083-855d-4ff4-9a3f-be933a02d32f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:52:02.387194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61789083-855d-4ff4-9a3f-be933a02d32f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:02.388598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61789083-855d-4ff4-9a3f-be933a02d32f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:02.397436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 573/1142] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:52:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:02.403610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dab60-9489-4fad-8010-eeb2814fcb9e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:02.518825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dab60-9489-4fad-8010-eeb2814fcb9e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:02.541387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:02.557839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce1548-dca7-4d56-9ef5-21f4fa725b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:02.558747 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce1548-dca7-4d56-9ef5-21f4fa725b54 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:02.559029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce1548-dca7-4d56-9ef5-21f4fa725b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:02.559305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce1548-dca7-4d56-9ef5-21f4fa725b54 None admin] PATH_INFO: `/v3/users/1602342ccf734c90b1aec0ed89b443ae` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:02.575377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce1548-dca7-4d56-9ef5-21f4fa725b54 None admin] RBAC: Authorizing `identity:get_user(user_id=1602342ccf734c90b1aec0ed89b443ae)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:02.576866 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce1548-dca7-4d56-9ef5-21f4fa725b54 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:02.578693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 570/1143] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:52:02 2022] GET /identity/v3/users/1602342ccf734c90b1aec0ed89b443ae => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:02.584978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61789083-855d-4ff4-9a3f-be933a02d32f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:02.712384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61789083-855d-4ff4-9a3f-be933a02d32f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:02.748018 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed13615-9996-4477-b2ee-3d8b2500380a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:02.748692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed13615-9996-4477-b2ee-3d8b2500380a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:02.748971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed13615-9996-4477-b2ee-3d8b2500380a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:02.749249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed13615-9996-4477-b2ee-3d8b2500380a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:02.803016 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:02.823032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed13615-9996-4477-b2ee-3d8b2500380a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:02.824557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed13615-9996-4477-b2ee-3d8b2500380a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:02.927899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 574/1144] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:52:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:52:02.934321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce1548-dca7-4d56-9ef5-21f4fa725b54 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:03.037653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce1548-dca7-4d56-9ef5-21f4fa725b54 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:03.080638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7430e-7610-4a55-866d-cd329ba024f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:03.081411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7430e-7610-4a55-866d-cd329ba024f5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:03.081736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7430e-7610-4a55-866d-cd329ba024f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:03.082015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7430e-7610-4a55-866d-cd329ba024f5 None admin] PATH_INFO: `/v3/projects/f0d4d86f3ad44532ba020a9cfee9dca3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:03.083226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7430e-7610-4a55-866d-cd329ba024f5 None admin] RBAC: Authorizing `identity:get_project(project_id=f0d4d86f3ad44532ba020a9cfee9dca3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:03.084823 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7430e-7610-4a55-866d-cd329ba024f5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:03.086560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 571/1145] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:52:02 2022] GET /identity/v3/projects/f0d4d86f3ad44532ba020a9cfee9dca3 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:03.092990 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed13615-9996-4477-b2ee-3d8b2500380a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:03.179413 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:03.201006 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed13615-9996-4477-b2ee-3d8b2500380a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:03.238157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f756a9-d9ad-4f6d-ae58-b42d1eca5c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:03.238826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f756a9-d9ad-4f6d-ae58-b42d1eca5c90 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:03.239102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f756a9-d9ad-4f6d-ae58-b42d1eca5c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:03.239383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f756a9-d9ad-4f6d-ae58-b42d1eca5c90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:03.240286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f756a9-d9ad-4f6d-ae58-b42d1eca5c90 None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=f0d4d86f3ad44532ba020a9cfee9dca3, user.id=1602342ccf734c90b1aec0ed89b443ae) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:52:03.240692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f756a9-d9ad-4f6d-ae58-b42d1eca5c90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:03.242174 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f756a9-d9ad-4f6d-ae58-b42d1eca5c90 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:03.248175 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 575/1146] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:52:03 2022] GET /identity/v3/role_assignments?user.id=1602342ccf734c90b1aec0ed89b443ae&scope.project.id=f0d4d86f3ad44532ba020a9cfee9dca3&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:05.249985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4ed4e-23b7-44d1-bfce-b9a69f1ff18a None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:05.250543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4ed4e-23b7-44d1-bfce-b9a69f1ff18a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:05.250543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4ed4e-23b7-44d1-bfce-b9a69f1ff18a None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:05.251990 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 572/1147] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:52:05.258034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19a490-4493-48d2-93e4-3c5f1865f3af None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:05.258528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19a490-4493-48d2-93e4-3c5f1865f3af None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:05.258528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19a490-4493-48d2-93e4-3c5f1865f3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:05.276889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9f19a490-4493-48d2-93e4-3c5f1865f3af None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:52:05.278177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9f19a490-4493-48d2-93e4-3c5f1865f3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:52:05.350346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 576/1148] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:52:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:52:05.373431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4ed4e-23b7-44d1-bfce-b9a69f1ff18a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:05.426782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:05.486664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4ed4e-23b7-44d1-bfce-b9a69f1ff18a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:05.529210 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f07b71-c60c-4963-a28a-cc1db33b5727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:05.530074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f07b71-c60c-4963-a28a-cc1db33b5727 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:05.530354 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f07b71-c60c-4963-a28a-cc1db33b5727 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:05.530631 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f07b71-c60c-4963-a28a-cc1db33b5727 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:05.543512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f07b71-c60c-4963-a28a-cc1db33b5727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a8f07b71-c60c-4963-a28a-cc1db33b5727 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:52:05.551394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:52:05.552583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:52:05.553526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 573/1149] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:52:05.559420 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19a490-4493-48d2-93e4-3c5f1865f3af None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:05.642997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:05.671746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19a490-4493-48d2-93e4-3c5f1865f3af None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:05.709074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0d829-4229-47c9-abb1-016cadd7497b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:05.709798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0d829-4229-47c9-abb1-016cadd7497b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:05.710089 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0d829-4229-47c9-abb1-016cadd7497b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:05.710360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0d829-4229-47c9-abb1-016cadd7497b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:05.711126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0d829-4229-47c9-abb1-016cadd7497b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:52:05.711433 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0d829-4229-47c9-abb1-016cadd7497b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:05.712834 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0d829-4229-47c9-abb1-016cadd7497b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:05.722426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 577/1150] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:52:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:05.728865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f07b71-c60c-4963-a28a-cc1db33b5727 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:05.840130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f07b71-c60c-4963-a28a-cc1db33b5727 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:05.847989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:05.877415 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e502a32-d94c-4708-8371-57cb9b6604f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:05.878333 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e502a32-d94c-4708-8371-57cb9b6604f2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:05.878615 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e502a32-d94c-4708-8371-57cb9b6604f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:05.878894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e502a32-d94c-4708-8371-57cb9b6604f2 None admin] PATH_INFO: `/v3/users/1602342ccf734c90b1aec0ed89b443ae` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:05.880484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e502a32-d94c-4708-8371-57cb9b6604f2 None admin] RBAC: Authorizing `identity:get_user(user_id=1602342ccf734c90b1aec0ed89b443ae)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:05.881950 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e502a32-d94c-4708-8371-57cb9b6604f2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:05.883669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 574/1151] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:52:05 2022] GET /identity/v3/users/1602342ccf734c90b1aec0ed89b443ae => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:05.889897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0d829-4229-47c9-abb1-016cadd7497b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:05.998827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0d829-4229-47c9-abb1-016cadd7497b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:06.038058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0979847-9c10-452f-9ed2-7bd46b15f635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:06.038931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0979847-9c10-452f-9ed2-7bd46b15f635 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:06.039213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0979847-9c10-452f-9ed2-7bd46b15f635 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:06.039487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0979847-9c10-452f-9ed2-7bd46b15f635 None admin] PATH_INFO: `/v3/projects/f0d4d86f3ad44532ba020a9cfee9dca3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:06.040732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0979847-9c10-452f-9ed2-7bd46b15f635 None admin] RBAC: Authorizing `identity:get_project(project_id=f0d4d86f3ad44532ba020a9cfee9dca3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:06.042235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0979847-9c10-452f-9ed2-7bd46b15f635 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:06.043981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 578/1152] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:52:05 2022] GET /identity/v3/projects/f0d4d86f3ad44532ba020a9cfee9dca3 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:06.050342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e502a32-d94c-4708-8371-57cb9b6604f2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:06.174825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e502a32-d94c-4708-8371-57cb9b6604f2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:06.211174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a60a68-2544-419a-9bec-6ad76411b9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:06.212084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a60a68-2544-419a-9bec-6ad76411b9cf None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:06.212359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a60a68-2544-419a-9bec-6ad76411b9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:06.212628 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a60a68-2544-419a-9bec-6ad76411b9cf None admin] PATH_INFO: `/v3/projects/f0d4d86f3ad44532ba020a9cfee9dca3/users/1602342ccf734c90b1aec0ed89b443ae/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:06.215238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a60a68-2544-419a-9bec-6ad76411b9cf None admin] RBAC: Authorizing `identity:create_grant(project_id=f0d4d86f3ad44532ba020a9cfee9dca3, user_id=1602342ccf734c90b1aec0ed89b443ae, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:06.216775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a60a68-2544-419a-9bec-6ad76411b9cf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:06.235410 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 575/1153] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:52:06 2022] PUT /identity/v3/projects/f0d4d86f3ad44532ba020a9cfee9dca3/users/1602342ccf734c90b1aec0ed89b443ae/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:52:08.261144 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694332fb-3231-476e-878b-ff5024b77d2c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:08.261733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694332fb-3231-476e-878b-ff5024b77d2c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:08.261733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694332fb-3231-476e-878b-ff5024b77d2c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:08.263233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 579/1154] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:52:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:52:08.269430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde48d59-e60f-41a6-99a4-8176ee9a7038 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:08.269965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde48d59-e60f-41a6-99a4-8176ee9a7038 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:08.270102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde48d59-e60f-41a6-99a4-8176ee9a7038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:08.290773 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cde48d59-e60f-41a6-99a4-8176ee9a7038 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:52:08.292588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cde48d59-e60f-41a6-99a4-8176ee9a7038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:52:08.324488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:08.376320 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 576/1155] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:52:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:52:08.400742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694332fb-3231-476e-878b-ff5024b77d2c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:08.405029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:08.529892 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694332fb-3231-476e-878b-ff5024b77d2c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:08.565527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212cbb1-81c1-4468-ad10-0db171f74d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:08.566370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212cbb1-81c1-4468-ad10-0db171f74d12 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:08.566654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212cbb1-81c1-4468-ad10-0db171f74d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:08.566929 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212cbb1-81c1-4468-ad10-0db171f74d12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:08.578779 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212cbb1-81c1-4468-ad10-0db171f74d12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f212cbb1-81c1-4468-ad10-0db171f74d12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:52:08.586473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:52:08.587485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:52:08.588487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 580/1156] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:52:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:52:08.594175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde48d59-e60f-41a6-99a4-8176ee9a7038 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:08.695576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde48d59-e60f-41a6-99a4-8176ee9a7038 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:08.731339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ccb62-8774-48ae-908d-307f4347681b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:08.732023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ccb62-8774-48ae-908d-307f4347681b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:08.732309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ccb62-8774-48ae-908d-307f4347681b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:08.732588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ccb62-8774-48ae-908d-307f4347681b None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:08.733377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ccb62-8774-48ae-908d-307f4347681b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:52:08.733677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ccb62-8774-48ae-908d-307f4347681b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:08.735133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ccb62-8774-48ae-908d-307f4347681b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:08.744272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 577/1157] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:52:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:08.750615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212cbb1-81c1-4468-ad10-0db171f74d12 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:08.820233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:08.859498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212cbb1-81c1-4468-ad10-0db171f74d12 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:08.894698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4427a5-e5fb-43a4-9c55-2f724beab8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:08.895567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4427a5-e5fb-43a4-9c55-2f724beab8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:08.895846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4427a5-e5fb-43a4-9c55-2f724beab8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:08.896197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4427a5-e5fb-43a4-9c55-2f724beab8d6 None admin] PATH_INFO: `/v3/users/1602342ccf734c90b1aec0ed89b443ae` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:08.897454 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4427a5-e5fb-43a4-9c55-2f724beab8d6 None admin] RBAC: Authorizing `identity:get_user(user_id=1602342ccf734c90b1aec0ed89b443ae)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:08.898931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4427a5-e5fb-43a4-9c55-2f724beab8d6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:08.900588 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 581/1158] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:52:08 2022] GET /identity/v3/users/1602342ccf734c90b1aec0ed89b443ae => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:08.906737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ccb62-8774-48ae-908d-307f4347681b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:08.936663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:09.010259 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ccb62-8774-48ae-908d-307f4347681b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:09.045060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc5344-ffbe-459e-8c44-906950f46a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:09.045752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc5344-ffbe-459e-8c44-906950f46a37 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:09.046041 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc5344-ffbe-459e-8c44-906950f46a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:09.046316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc5344-ffbe-459e-8c44-906950f46a37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:09.125875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc5344-ffbe-459e-8c44-906950f46a37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:09.127351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc5344-ffbe-459e-8c44-906950f46a37 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:09.158612 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:09.236908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 578/1159] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:52:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:52:09.243301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4427a5-e5fb-43a4-9c55-2f724beab8d6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:09.348653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4427a5-e5fb-43a4-9c55-2f724beab8d6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:09.379000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:09.385511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910aab1-366d-4c99-adad-690fdc851420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:09.386414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910aab1-366d-4c99-adad-690fdc851420 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:09.386689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910aab1-366d-4c99-adad-690fdc851420 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:09.386959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910aab1-366d-4c99-adad-690fdc851420 None admin] PATH_INFO: `/v3/projects/f0d4d86f3ad44532ba020a9cfee9dca3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:09.388187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910aab1-366d-4c99-adad-690fdc851420 None admin] RBAC: Authorizing `identity:get_project(project_id=f0d4d86f3ad44532ba020a9cfee9dca3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:09.389652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910aab1-366d-4c99-adad-690fdc851420 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:09.391370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 582/1160] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:52:09 2022] GET /identity/v3/projects/f0d4d86f3ad44532ba020a9cfee9dca3 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:09.398127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc5344-ffbe-459e-8c44-906950f46a37 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:09.502706 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc5344-ffbe-459e-8c44-906950f46a37 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:09.530223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:09.540147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6024c-9a97-481f-8e01-5cfab975550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:09.540813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6024c-9a97-481f-8e01-5cfab975550c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:09.541097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6024c-9a97-481f-8e01-5cfab975550c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:09.541357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6024c-9a97-481f-8e01-5cfab975550c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:09.542228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6024c-9a97-481f-8e01-5cfab975550c None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=f0d4d86f3ad44532ba020a9cfee9dca3, user.id=1602342ccf734c90b1aec0ed89b443ae) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:52:09.542538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6024c-9a97-481f-8e01-5cfab975550c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:09.543982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6024c-9a97-481f-8e01-5cfab975550c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:09.550390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 579/1161] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:52:09 2022] GET /identity/v3/role_assignments?user.id=1602342ccf734c90b1aec0ed89b443ae&scope.project.id=f0d4d86f3ad44532ba020a9cfee9dca3&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:11.576922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e1faf-7f30-4836-ace5-70cb65a3000b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:11.577541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e1faf-7f30-4836-ace5-70cb65a3000b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:11.577541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e1faf-7f30-4836-ace5-70cb65a3000b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:11.579146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 583/1162] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:52:11.584282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d73a6-495b-4b80-a001-8fc93462ce0a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:11.584743 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d73a6-495b-4b80-a001-8fc93462ce0a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:11.584845 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d73a6-495b-4b80-a001-8fc93462ce0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:11.602062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f45d73a6-495b-4b80-a001-8fc93462ce0a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:52:11.603255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f45d73a6-495b-4b80-a001-8fc93462ce0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:52:11.672758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 580/1163] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:52:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:52:11.695436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e1faf-7f30-4836-ace5-70cb65a3000b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:11.805039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e1faf-7f30-4836-ace5-70cb65a3000b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:11.841375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99138acd-de08-46d1-b1ca-20eb63c72261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:11.841963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99138acd-de08-46d1-b1ca-20eb63c72261 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:11.842244 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99138acd-de08-46d1-b1ca-20eb63c72261 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:11.842527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99138acd-de08-46d1-b1ca-20eb63c72261 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:11.911997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:11.915053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99138acd-de08-46d1-b1ca-20eb63c72261 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:11.916665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99138acd-de08-46d1-b1ca-20eb63c72261 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:12.017714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 584/1164] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:52:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:52:12.024304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d73a6-495b-4b80-a001-8fc93462ce0a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:12.131133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d73a6-495b-4b80-a001-8fc93462ce0a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:12.152316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:12.168157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4fc62-b065-44e7-ace7-67d86efcd4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:12.168966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4fc62-b065-44e7-ace7-67d86efcd4eb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:12.169244 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4fc62-b065-44e7-ace7-67d86efcd4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:12.169519 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4fc62-b065-44e7-ace7-67d86efcd4eb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:12.176480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4fc62-b065-44e7-ace7-67d86efcd4eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:12.178296 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4fc62-b065-44e7-ace7-67d86efcd4eb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1af4fc62-b065-44e7-ace7-67d86efcd4eb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:52:12.193386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:52:12.194756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:52:12.195644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 581/1165] 10.222.0.43 () {56 vars in 1117 bytes} [Mon Jul 4 07:52:12 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:52:12.201140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99138acd-de08-46d1-b1ca-20eb63c72261 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:12.303898 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:12.307188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99138acd-de08-46d1-b1ca-20eb63c72261 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:12.347139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567ccdb-899e-4fda-b325-875739364f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:12.347821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567ccdb-899e-4fda-b325-875739364f00 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:12.348100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567ccdb-899e-4fda-b325-875739364f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:12.348379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567ccdb-899e-4fda-b325-875739364f00 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:12.349161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567ccdb-899e-4fda-b325-875739364f00 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:52:12.349737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567ccdb-899e-4fda-b325-875739364f00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:12.351190 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567ccdb-899e-4fda-b325-875739364f00 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:12.359609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 585/1166] 10.222.0.43 () {56 vars in 1115 bytes} [Mon Jul 4 07:52:12 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:14.393266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27db12-476c-437d-a625-8b65032f48eb None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:14.393841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27db12-476c-437d-a625-8b65032f48eb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:14.393841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27db12-476c-437d-a625-8b65032f48eb None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:14.395166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 582/1167] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:52:14.400739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecc6c8-2d0e-42c6-9bf8-b23aaf6ddfa6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:14.401195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecc6c8-2d0e-42c6-9bf8-b23aaf6ddfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:14.401287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecc6c8-2d0e-42c6-9bf8-b23aaf6ddfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:14.419667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d6ecc6c8-2d0e-42c6-9bf8-b23aaf6ddfa6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:52:14.420897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d6ecc6c8-2d0e-42c6-9bf8-b23aaf6ddfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:52:14.495560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 586/1168] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:52:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:52:14.519302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27db12-476c-437d-a625-8b65032f48eb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:14.639634 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27db12-476c-437d-a625-8b65032f48eb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:14.675488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70f722-ef6b-4c1a-a794-b75dc4e534e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:14.676071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70f722-ef6b-4c1a-a794-b75dc4e534e9 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:14.676353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70f722-ef6b-4c1a-a794-b75dc4e534e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:14.676753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70f722-ef6b-4c1a-a794-b75dc4e534e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:14.677924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70f722-ef6b-4c1a-a794-b75dc4e534e9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:14.679121 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70f722-ef6b-4c1a-a794-b75dc4e534e9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:14.697587 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 583/1169] 10.222.0.43 () {60 vars in 1138 bytes} [Mon Jul 4 07:52:14 2022] POST /identity/v3/domains => generated 267 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:52:16.759326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c16fca-0b9a-48b8-833b-e63e6619bc11 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:16.759919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c16fca-0b9a-48b8-833b-e63e6619bc11 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:16.759919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c16fca-0b9a-48b8-833b-e63e6619bc11 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:16.761233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 587/1170] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:52:16.766674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3908a-76d3-4474-98d6-7960ebca3e1f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:16.767115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3908a-76d3-4474-98d6-7960ebca3e1f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:16.767215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3908a-76d3-4474-98d6-7960ebca3e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:16.785303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-40d3908a-76d3-4474-98d6-7960ebca3e1f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:52:16.786849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-40d3908a-76d3-4474-98d6-7960ebca3e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:52:16.824359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:16.862001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 584/1171] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:52:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:52:16.891583 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c16fca-0b9a-48b8-833b-e63e6619bc11 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:17.002221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:17.012618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c16fca-0b9a-48b8-833b-e63e6619bc11 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:17.052087 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f16dc-7d83-46fc-b77c-53bf65a69710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:17.053665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f16dc-7d83-46fc-b77c-53bf65a69710 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:17.053941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f16dc-7d83-46fc-b77c-53bf65a69710 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:17.054211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f16dc-7d83-46fc-b77c-53bf65a69710 None admin] PATH_INFO: `/v3/domains/123632d647b94a7ab974616f83696885` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:17.066666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f16dc-7d83-46fc-b77c-53bf65a69710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=123632d647b94a7ab974616f83696885)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:17.068750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f16dc-7d83-46fc-b77c-53bf65a69710 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:17.070635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 588/1172] 10.222.0.43 () {56 vars in 1183 bytes} [Mon Jul 4 07:52:16 2022] GET /identity/v3/domains/123632d647b94a7ab974616f83696885 => generated 267 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:17.076971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3908a-76d3-4474-98d6-7960ebca3e1f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:17.190004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3908a-76d3-4474-98d6-7960ebca3e1f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:17.229700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78c01f-b94b-4268-ba7e-4b46f3287373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:17.230407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78c01f-b94b-4268-ba7e-4b46f3287373 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:17.230736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78c01f-b94b-4268-ba7e-4b46f3287373 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:17.231013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78c01f-b94b-4268-ba7e-4b46f3287373 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:17.231754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78c01f-b94b-4268-ba7e-4b46f3287373 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:17.233200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78c01f-b94b-4268-ba7e-4b46f3287373 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:17.263484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 585/1173] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:52:17 2022] POST /identity/v3/projects => generated 361 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:52:19.378718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6a788-24a2-4bd7-bd6a-f4ae63ec9a30 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:19.379274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6a788-24a2-4bd7-bd6a-f4ae63ec9a30 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:19.379274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6a788-24a2-4bd7-bd6a-f4ae63ec9a30 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:19.381018 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 589/1174] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:52:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:52:19.387382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f4a56-2f76-479f-b451-305bcb041df3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:19.387820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f4a56-2f76-479f-b451-305bcb041df3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:19.387962 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f4a56-2f76-479f-b451-305bcb041df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:19.410367 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-838f4a56-2f76-479f-b451-305bcb041df3 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:52:19.411810 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-838f4a56-2f76-479f-b451-305bcb041df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:52:19.431047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:19.491396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 586/1175] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:52:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:52:19.516623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6a788-24a2-4bd7-bd6a-f4ae63ec9a30 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:19.634828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6a788-24a2-4bd7-bd6a-f4ae63ec9a30 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:19.671618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7b1eb-0a6c-445c-8f2c-fd7dbdaf96e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:19.672695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7b1eb-0a6c-445c-8f2c-fd7dbdaf96e2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:19.673140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7b1eb-0a6c-445c-8f2c-fd7dbdaf96e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:19.673529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7b1eb-0a6c-445c-8f2c-fd7dbdaf96e2 None admin] PATH_INFO: `/v3/domains/123632d647b94a7ab974616f83696885` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:19.674904 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7b1eb-0a6c-445c-8f2c-fd7dbdaf96e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=123632d647b94a7ab974616f83696885)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:19.676369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7b1eb-0a6c-445c-8f2c-fd7dbdaf96e2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:19.678503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 590/1176] 10.222.0.43 () {56 vars in 1183 bytes} [Mon Jul 4 07:52:19 2022] GET /identity/v3/domains/123632d647b94a7ab974616f83696885 => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:19.686041 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f4a56-2f76-479f-b451-305bcb041df3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:19.796554 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f4a56-2f76-479f-b451-305bcb041df3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:19.844302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85009a-f3d3-44fd-8fff-f8d476bf9007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:19.845242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85009a-f3d3-44fd-8fff-f8d476bf9007 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:19.845531 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85009a-f3d3-44fd-8fff-f8d476bf9007 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:19.845909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85009a-f3d3-44fd-8fff-f8d476bf9007 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:19.846636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85009a-f3d3-44fd-8fff-f8d476bf9007 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:19.848079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85009a-f3d3-44fd-8fff-f8d476bf9007 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:19.876593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 587/1177] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:52:19 2022] POST /identity/v3/users => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 07:52:22.004127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f66f0-6d9d-4acb-9eff-1e4c0774febc None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:22.005107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f66f0-6d9d-4acb-9eff-1e4c0774febc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:22.005107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f66f0-6d9d-4acb-9eff-1e4c0774febc None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:22.006622 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 591/1178] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:52:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:52:22.012496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98ba5f-91fe-4a13-b06d-ea629d0fed34 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:22.013051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98ba5f-91fe-4a13-b06d-ea629d0fed34 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:22.013051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98ba5f-91fe-4a13-b06d-ea629d0fed34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:22.031075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fb98ba5f-91fe-4a13-b06d-ea629d0fed34 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:52:22.032298 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fb98ba5f-91fe-4a13-b06d-ea629d0fed34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:52:22.034503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:22.118993 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 588/1179] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:52:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:52:22.143128 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f66f0-6d9d-4acb-9eff-1e4c0774febc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:22.171899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:22.258656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f66f0-6d9d-4acb-9eff-1e4c0774febc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:22.295546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd501b-eb9b-4f4b-9896-7cea566ea1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:22.296350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd501b-eb9b-4f4b-9896-7cea566ea1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:22.296633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd501b-eb9b-4f4b-9896-7cea566ea1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:22.296911 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd501b-eb9b-4f4b-9896-7cea566ea1e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:22.309300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd501b-eb9b-4f4b-9896-7cea566ea1e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9ccd501b-eb9b-4f4b-9896-7cea566ea1e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:52:22.317050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:52:22.318959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:52:22.320534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 592/1180] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:52:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:52:22.329161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98ba5f-91fe-4a13-b06d-ea629d0fed34 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:22.442737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98ba5f-91fe-4a13-b06d-ea629d0fed34 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:22.494399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdccf8-1c6b-4442-afb3-7f3d210037b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:22.495395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdccf8-1c6b-4442-afb3-7f3d210037b4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:22.495868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdccf8-1c6b-4442-afb3-7f3d210037b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:22.496354 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdccf8-1c6b-4442-afb3-7f3d210037b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:22.497543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdccf8-1c6b-4442-afb3-7f3d210037b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:52:22.498107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdccf8-1c6b-4442-afb3-7f3d210037b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:22.500414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdccf8-1c6b-4442-afb3-7f3d210037b4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:22.510686 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:22.517399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 589/1181] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:52:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:22.526248 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd501b-eb9b-4f4b-9896-7cea566ea1e7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:22.620640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:22.640970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd501b-eb9b-4f4b-9896-7cea566ea1e7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:22.678531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99f2d8-ca06-4aeb-9fe9-4df63a2a2b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:22.679412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99f2d8-ca06-4aeb-9fe9-4df63a2a2b4d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:22.679694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99f2d8-ca06-4aeb-9fe9-4df63a2a2b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:22.679965 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99f2d8-ca06-4aeb-9fe9-4df63a2a2b4d None admin] PATH_INFO: `/v3/users/df0061be33a444c7806b3d23f695ccae` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:22.694973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99f2d8-ca06-4aeb-9fe9-4df63a2a2b4d None admin] RBAC: Authorizing `identity:get_user(user_id=df0061be33a444c7806b3d23f695ccae)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:22.696580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99f2d8-ca06-4aeb-9fe9-4df63a2a2b4d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:22.698388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 593/1182] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:52:22 2022] GET /identity/v3/users/df0061be33a444c7806b3d23f695ccae => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:22.705132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdccf8-1c6b-4442-afb3-7f3d210037b4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:22.855397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdccf8-1c6b-4442-afb3-7f3d210037b4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:22.893935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff0306-7f88-4fc9-8950-65f79a2da309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:22.894688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff0306-7f88-4fc9-8950-65f79a2da309 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:22.894973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff0306-7f88-4fc9-8950-65f79a2da309 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:22.895250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff0306-7f88-4fc9-8950-65f79a2da309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:22.975213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:22.978342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff0306-7f88-4fc9-8950-65f79a2da309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:22.979970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff0306-7f88-4fc9-8950-65f79a2da309 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:23.118861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 590/1183] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:52:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:52:23.126417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99f2d8-ca06-4aeb-9fe9-4df63a2a2b4d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:23.273645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99f2d8-ca06-4aeb-9fe9-4df63a2a2b4d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:23.315775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751348a-56f8-417e-b69e-77cb730d3806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:23.316651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751348a-56f8-417e-b69e-77cb730d3806 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:23.316931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751348a-56f8-417e-b69e-77cb730d3806 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:23.317203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751348a-56f8-417e-b69e-77cb730d3806 None admin] PATH_INFO: `/v3/projects/0f933dc138254df78dbbaa2cb3134939` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:23.318875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2751348a-56f8-417e-b69e-77cb730d3806 None admin] RBAC: Authorizing `identity:get_project(project_id=0f933dc138254df78dbbaa2cb3134939)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:23.320357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2751348a-56f8-417e-b69e-77cb730d3806 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:23.322215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 594/1184] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:52:23 2022] GET /identity/v3/projects/0f933dc138254df78dbbaa2cb3134939 => generated 361 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:23.329029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff0306-7f88-4fc9-8950-65f79a2da309 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:23.455459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:23.458409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff0306-7f88-4fc9-8950-65f79a2da309 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:23.496189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bd6a6-b1f0-4d4d-9a44-836adfd1ddbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:23.496855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bd6a6-b1f0-4d4d-9a44-836adfd1ddbb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:23.497134 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bd6a6-b1f0-4d4d-9a44-836adfd1ddbb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:23.497416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bd6a6-b1f0-4d4d-9a44-836adfd1ddbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:23.498291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bd6a6-b1f0-4d4d-9a44-836adfd1ddbb None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=0f933dc138254df78dbbaa2cb3134939, user.id=df0061be33a444c7806b3d23f695ccae) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:52:23.498604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bd6a6-b1f0-4d4d-9a44-836adfd1ddbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:23.500060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bd6a6-b1f0-4d4d-9a44-836adfd1ddbb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:23.506657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 591/1185] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:52:23 2022] GET /identity/v3/role_assignments?user.id=df0061be33a444c7806b3d23f695ccae&scope.project.id=0f933dc138254df78dbbaa2cb3134939&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:25.842157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fb537-c171-425f-b3a6-c72aaa6a1254 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:25.842871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fb537-c171-425f-b3a6-c72aaa6a1254 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:25.842871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fb537-c171-425f-b3a6-c72aaa6a1254 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:25.844156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 595/1186] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:52:25.851913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d78e5-b5dc-472d-bc2b-81974783c60b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:25.852483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d78e5-b5dc-472d-bc2b-81974783c60b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:25.852889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d78e5-b5dc-472d-bc2b-81974783c60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:25.881468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ca2d78e5-b5dc-472d-bc2b-81974783c60b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:52:25.883457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ca2d78e5-b5dc-472d-bc2b-81974783c60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:52:25.979703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 592/1187] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:52:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:52:26.006715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fb537-c171-425f-b3a6-c72aaa6a1254 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:26.019246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:26.147029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fb537-c171-425f-b3a6-c72aaa6a1254 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:26.189976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb571b-14c3-4438-9797-690ac58de30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:26.191300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb571b-14c3-4438-9797-690ac58de30e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:26.191592 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb571b-14c3-4438-9797-690ac58de30e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:26.191872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb571b-14c3-4438-9797-690ac58de30e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:26.205124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb571b-14c3-4438-9797-690ac58de30e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-46eb571b-14c3-4438-9797-690ac58de30e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:52:26.213206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:52:26.214383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:52:26.215571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 596/1188] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:52:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:52:26.221160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d78e5-b5dc-472d-bc2b-81974783c60b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:26.330981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:26.338601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d78e5-b5dc-472d-bc2b-81974783c60b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:26.376673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead86227-8b22-446a-8c10-0a34b7ded2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:26.377286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead86227-8b22-446a-8c10-0a34b7ded2eb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:26.377799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead86227-8b22-446a-8c10-0a34b7ded2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:26.378082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead86227-8b22-446a-8c10-0a34b7ded2eb None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:26.378856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead86227-8b22-446a-8c10-0a34b7ded2eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:52:26.379167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead86227-8b22-446a-8c10-0a34b7ded2eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:26.381183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead86227-8b22-446a-8c10-0a34b7ded2eb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:26.390830 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 593/1189] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:52:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:26.399713 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb571b-14c3-4438-9797-690ac58de30e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:26.473798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:26.515910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb571b-14c3-4438-9797-690ac58de30e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:26.555392 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78d09a-4aac-4820-837f-54b708e235d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:26.556285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78d09a-4aac-4820-837f-54b708e235d0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:26.556564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78d09a-4aac-4820-837f-54b708e235d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:26.556839 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78d09a-4aac-4820-837f-54b708e235d0 None admin] PATH_INFO: `/v3/users/df0061be33a444c7806b3d23f695ccae` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:26.558129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78d09a-4aac-4820-837f-54b708e235d0 None admin] RBAC: Authorizing `identity:get_user(user_id=df0061be33a444c7806b3d23f695ccae)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:26.559605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78d09a-4aac-4820-837f-54b708e235d0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:26.561376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 597/1190] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:52:26 2022] GET /identity/v3/users/df0061be33a444c7806b3d23f695ccae => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:26.567592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead86227-8b22-446a-8c10-0a34b7ded2eb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:26.688429 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead86227-8b22-446a-8c10-0a34b7ded2eb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:26.730316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f3070-0e35-4538-8d78-087fae62400d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:26.731228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f3070-0e35-4538-8d78-087fae62400d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:26.731512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f3070-0e35-4538-8d78-087fae62400d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:26.731789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f3070-0e35-4538-8d78-087fae62400d None admin] PATH_INFO: `/v3/projects/0f933dc138254df78dbbaa2cb3134939` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:26.733280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f3070-0e35-4538-8d78-087fae62400d None admin] RBAC: Authorizing `identity:get_project(project_id=0f933dc138254df78dbbaa2cb3134939)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:26.734943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f3070-0e35-4538-8d78-087fae62400d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:26.737016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 594/1191] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:52:26 2022] GET /identity/v3/projects/0f933dc138254df78dbbaa2cb3134939 => generated 361 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:26.744589 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78d09a-4aac-4820-837f-54b708e235d0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:26.887671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78d09a-4aac-4820-837f-54b708e235d0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:26.943877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b44f98-a724-4db0-b598-5215cc8e8729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:26.944944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b44f98-a724-4db0-b598-5215cc8e8729 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:26.945280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b44f98-a724-4db0-b598-5215cc8e8729 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:26.945602 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b44f98-a724-4db0-b598-5215cc8e8729 None admin] PATH_INFO: `/v3/projects/0f933dc138254df78dbbaa2cb3134939/users/df0061be33a444c7806b3d23f695ccae/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:26.948929 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b44f98-a724-4db0-b598-5215cc8e8729 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f933dc138254df78dbbaa2cb3134939, user_id=df0061be33a444c7806b3d23f695ccae, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:26.951760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b44f98-a724-4db0-b598-5215cc8e8729 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:26.957276 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:26.971601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 598/1192] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:52:26 2022] PUT /identity/v3/projects/0f933dc138254df78dbbaa2cb3134939/users/df0061be33a444c7806b3d23f695ccae/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:52:29.077495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1d020-415f-4507-a740-d537082bc525 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:29.078163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1d020-415f-4507-a740-d537082bc525 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:29.078464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1d020-415f-4507-a740-d537082bc525 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:29.081077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 595/1193] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:52:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:52:29.087519 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37645a54-e93a-489e-be3d-b3bd86314759 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:29.088118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37645a54-e93a-489e-be3d-b3bd86314759 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:29.088118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37645a54-e93a-489e-be3d-b3bd86314759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:29.107008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-37645a54-e93a-489e-be3d-b3bd86314759 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:52:29.108680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-37645a54-e93a-489e-be3d-b3bd86314759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:52:29.187505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 599/1194] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:52:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:52:29.212264 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1d020-415f-4507-a740-d537082bc525 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:29.244014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:29.334300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1d020-415f-4507-a740-d537082bc525 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:29.373917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac4f4b-bdd2-4b28-9e94-99d2c9f9838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:29.374712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac4f4b-bdd2-4b28-9e94-99d2c9f9838a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:29.374993 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac4f4b-bdd2-4b28-9e94-99d2c9f9838a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:29.375265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac4f4b-bdd2-4b28-9e94-99d2c9f9838a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:29.388478 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ac4f4b-bdd2-4b28-9e94-99d2c9f9838a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c9ac4f4b-bdd2-4b28-9e94-99d2c9f9838a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:52:29.397377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:52:29.398673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:52:29.399614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 596/1195] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:52:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:52:29.405317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37645a54-e93a-489e-be3d-b3bd86314759 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:29.531806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37645a54-e93a-489e-be3d-b3bd86314759 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:29.562257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:29.573038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8412526-1bac-4ffc-8aa4-a0dc7aae69b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:29.573874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8412526-1bac-4ffc-8aa4-a0dc7aae69b0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:29.574217 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8412526-1bac-4ffc-8aa4-a0dc7aae69b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:29.574546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8412526-1bac-4ffc-8aa4-a0dc7aae69b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:29.575357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8412526-1bac-4ffc-8aa4-a0dc7aae69b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:52:29.575847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8412526-1bac-4ffc-8aa4-a0dc7aae69b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:29.577333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8412526-1bac-4ffc-8aa4-a0dc7aae69b0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:29.587028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 600/1196] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:52:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:29.593951 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac4f4b-bdd2-4b28-9e94-99d2c9f9838a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:29.687460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:29.704922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac4f4b-bdd2-4b28-9e94-99d2c9f9838a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:29.741558 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bfe72-eee6-4d4b-bfaf-15e5789bda2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:29.742503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bfe72-eee6-4d4b-bfaf-15e5789bda2d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:29.742780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bfe72-eee6-4d4b-bfaf-15e5789bda2d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:29.743052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bfe72-eee6-4d4b-bfaf-15e5789bda2d None admin] PATH_INFO: `/v3/users/df0061be33a444c7806b3d23f695ccae` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:29.744439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bfe72-eee6-4d4b-bfaf-15e5789bda2d None admin] RBAC: Authorizing `identity:get_user(user_id=df0061be33a444c7806b3d23f695ccae)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:29.746048 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bfe72-eee6-4d4b-bfaf-15e5789bda2d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:29.747842 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 597/1197] 10.222.0.43 () {56 vars in 1177 bytes} [Mon Jul 4 07:52:29 2022] GET /identity/v3/users/df0061be33a444c7806b3d23f695ccae => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:29.755040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8412526-1bac-4ffc-8aa4-a0dc7aae69b0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:29.871615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8412526-1bac-4ffc-8aa4-a0dc7aae69b0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:29.908405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15832611-734c-4d2f-8879-c0a2b915b7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:29.909051 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15832611-734c-4d2f-8879-c0a2b915b7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:29.909331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15832611-734c-4d2f-8879-c0a2b915b7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:29.909599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15832611-734c-4d2f-8879-c0a2b915b7f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:29.971059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:29.985246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15832611-734c-4d2f-8879-c0a2b915b7f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:29.986752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15832611-734c-4d2f-8879-c0a2b915b7f9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:30.098054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 601/1198] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:52:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:52:30.105238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bfe72-eee6-4d4b-bfaf-15e5789bda2d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:30.247472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bfe72-eee6-4d4b-bfaf-15e5789bda2d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:30.287091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11eb5ab-6d20-4f19-a99c-2eeb7e903582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:30.288004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11eb5ab-6d20-4f19-a99c-2eeb7e903582 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:30.288288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11eb5ab-6d20-4f19-a99c-2eeb7e903582 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:30.288564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11eb5ab-6d20-4f19-a99c-2eeb7e903582 None admin] PATH_INFO: `/v3/projects/0f933dc138254df78dbbaa2cb3134939` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:30.289896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11eb5ab-6d20-4f19-a99c-2eeb7e903582 None admin] RBAC: Authorizing `identity:get_project(project_id=0f933dc138254df78dbbaa2cb3134939)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:30.291395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11eb5ab-6d20-4f19-a99c-2eeb7e903582 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:30.293236 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 598/1199] 10.222.0.43 () {56 vars in 1186 bytes} [Mon Jul 4 07:52:30 2022] GET /identity/v3/projects/0f933dc138254df78dbbaa2cb3134939 => generated 361 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:30.300574 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15832611-734c-4d2f-8879-c0a2b915b7f9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:52:30.414909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:30.429486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15832611-734c-4d2f-8879-c0a2b915b7f9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:52:30.468806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279b054-cc34-494b-9231-7a79b5a8d2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:52:30.469479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279b054-cc34-494b-9231-7a79b5a8d2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:52:30.469802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279b054-cc34-494b-9231-7a79b5a8d2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:52:30.470083 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279b054-cc34-494b-9231-7a79b5a8d2a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:52:30.470955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279b054-cc34-494b-9231-7a79b5a8d2a4 None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=0f933dc138254df78dbbaa2cb3134939, user.id=df0061be33a444c7806b3d23f695ccae) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:52:30.471266 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279b054-cc34-494b-9231-7a79b5a8d2a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:52:30.472755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279b054-cc34-494b-9231-7a79b5a8d2a4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:52:30.479714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 602/1200] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:52:30 2022] GET /identity/v3/role_assignments?user.id=df0061be33a444c7806b3d23f695ccae&scope.project.id=0f933dc138254df78dbbaa2cb3134939&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:52:40.282433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:52:40.476863 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=24 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:28.826516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b644901-b6b3-4e31-8851-c0a43d0284b4 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:28.827135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b644901-b6b3-4e31-8851-c0a43d0284b4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:28.827135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b644901-b6b3-4e31-8851-c0a43d0284b4 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:28.828673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 599/1201] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:53:28.835065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac4554-b4ff-4123-b78b-112dd272d27e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:28.835638 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac4554-b4ff-4123-b78b-112dd272d27e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:28.835737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac4554-b4ff-4123-b78b-112dd272d27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:28.854833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c0ac4554-b4ff-4123-b78b-112dd272d27e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:53:28.856420 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c0ac4554-b4ff-4123-b78b-112dd272d27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:53:28.946803 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 603/1202] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:53:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:53:28.978228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b644901-b6b3-4e31-8851-c0a43d0284b4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:29.112387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b644901-b6b3-4e31-8851-c0a43d0284b4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:29.157996 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c278840-bb6b-4c76-8fa1-35a0f214f66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:29.159266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c278840-bb6b-4c76-8fa1-35a0f214f66a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:29.159759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c278840-bb6b-4c76-8fa1-35a0f214f66a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:29.160255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c278840-bb6b-4c76-8fa1-35a0f214f66a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:29.174866 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c278840-bb6b-4c76-8fa1-35a0f214f66a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:29.177017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c278840-bb6b-4c76-8fa1-35a0f214f66a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1c278840-bb6b-4c76-8fa1-35a0f214f66a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:29.203538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:29.205016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:29.206497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:29.206497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:29.206497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:29.206497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:29.206497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 600/1203] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:53:29.211771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac4554-b4ff-4123-b78b-112dd272d27e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:29.336249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac4554-b4ff-4123-b78b-112dd272d27e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:29.398352 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:29.406453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f0a0c-113f-4c81-95f1-474e8f4a57f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:29.407171 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0a0c-113f-4c81-95f1-474e8f4a57f8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:29.407465 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0a0c-113f-4c81-95f1-474e8f4a57f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:29.407751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0a0c-113f-4c81-95f1-474e8f4a57f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:29.408597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f0a0c-113f-4c81-95f1-474e8f4a57f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:29.408864 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f0a0c-113f-4c81-95f1-474e8f4a57f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:29.410797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f0a0c-113f-4c81-95f1-474e8f4a57f8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:29.427393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 604/1204] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:29.433711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c278840-bb6b-4c76-8fa1-35a0f214f66a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:29.506355 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:29.576521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c278840-bb6b-4c76-8fa1-35a0f214f66a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:29.623686 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69828f7c-efd4-40fb-b736-f7bc13402949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:29.624786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69828f7c-efd4-40fb-b736-f7bc13402949 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:29.625263 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69828f7c-efd4-40fb-b736-f7bc13402949 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:29.625745 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69828f7c-efd4-40fb-b736-f7bc13402949 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:29.626677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69828f7c-efd4-40fb-b736-f7bc13402949 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:29.628205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69828f7c-efd4-40fb-b736-f7bc13402949 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:29.665745 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 601/1205] 10.222.0.43 () {60 vars in 1132 bytes} [Mon Jul 4 07:53:29 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:53:31.875498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e820ce-e63c-44ff-bb88-6f54cd507e55 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:31.876078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e820ce-e63c-44ff-bb88-6f54cd507e55 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:31.876078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e820ce-e63c-44ff-bb88-6f54cd507e55 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:31.877451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 605/1206] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:53:31.883613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf8e25-0e14-4973-9104-9ed145ebd6ee None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:31.884445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf8e25-0e14-4973-9104-9ed145ebd6ee None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:31.884445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf8e25-0e14-4973-9104-9ed145ebd6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:31.901987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-02cf8e25-0e14-4973-9104-9ed145ebd6ee None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:53:31.903203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-02cf8e25-0e14-4973-9104-9ed145ebd6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:53:32.014302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 602/1207] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:53:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:53:32.047637 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e820ce-e63c-44ff-bb88-6f54cd507e55 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:32.193338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e820ce-e63c-44ff-bb88-6f54cd507e55 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:32.244482 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d563dc-ec34-4303-88e5-0d4b01e89012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:32.245392 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d563dc-ec34-4303-88e5-0d4b01e89012 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:32.245725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d563dc-ec34-4303-88e5-0d4b01e89012 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:32.246078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d563dc-ec34-4303-88e5-0d4b01e89012 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:32.264442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d563dc-ec34-4303-88e5-0d4b01e89012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-02d563dc-ec34-4303-88e5-0d4b01e89012 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:53:32.274587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:32.275714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:32.282382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 606/1208] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:53:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:32.284780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf8e25-0e14-4973-9104-9ed145ebd6ee None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:32.337040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:32.421560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf8e25-0e14-4973-9104-9ed145ebd6ee None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:32.472636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065791b-c3eb-4c08-addc-42373f65024b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:32.473306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065791b-c3eb-4c08-addc-42373f65024b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:32.473592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065791b-c3eb-4c08-addc-42373f65024b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:32.473999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065791b-c3eb-4c08-addc-42373f65024b None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:32.474771 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065791b-c3eb-4c08-addc-42373f65024b None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:32.475081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065791b-c3eb-4c08-addc-42373f65024b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:32.476490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065791b-c3eb-4c08-addc-42373f65024b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:32.485919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 603/1209] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:53:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:32.492155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d563dc-ec34-4303-88e5-0d4b01e89012 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:32.560839 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:32.642124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d563dc-ec34-4303-88e5-0d4b01e89012 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:32.685390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68936a9-f07f-4dec-b9ff-47b760e735cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:32.686346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68936a9-f07f-4dec-b9ff-47b760e735cd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:32.686614 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68936a9-f07f-4dec-b9ff-47b760e735cd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:32.686897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68936a9-f07f-4dec-b9ff-47b760e735cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:32.701485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68936a9-f07f-4dec-b9ff-47b760e735cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:32.703105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68936a9-f07f-4dec-b9ff-47b760e735cd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d68936a9-f07f-4dec-b9ff-47b760e735cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:32.725329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:32.726554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:32.727606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:32.727606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:32.727606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:32.727606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:32.727606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 607/1210] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:53:32.733243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065791b-c3eb-4c08-addc-42373f65024b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:32.887580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:32.891206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065791b-c3eb-4c08-addc-42373f65024b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:32.939876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26c13a-11d7-4761-8a68-cdd94ff04934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:32.940619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26c13a-11d7-4761-8a68-cdd94ff04934 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:32.940928 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26c13a-11d7-4761-8a68-cdd94ff04934 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:32.941316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26c13a-11d7-4761-8a68-cdd94ff04934 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:32.942553 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26c13a-11d7-4761-8a68-cdd94ff04934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:32.943118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26c13a-11d7-4761-8a68-cdd94ff04934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:32.944925 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26c13a-11d7-4761-8a68-cdd94ff04934 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:32.963587 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 604/1211] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:32.972025 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68936a9-f07f-4dec-b9ff-47b760e735cd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:33.120883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68936a9-f07f-4dec-b9ff-47b760e735cd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:33.123089 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:33.168283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982a0c2-cdb8-4822-ac42-dc795c4a27ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:33.169176 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982a0c2-cdb8-4822-ac42-dc795c4a27ed None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:33.169486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982a0c2-cdb8-4822-ac42-dc795c4a27ed None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:33.169920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982a0c2-cdb8-4822-ac42-dc795c4a27ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:33.181312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982a0c2-cdb8-4822-ac42-dc795c4a27ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:33.183017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982a0c2-cdb8-4822-ac42-dc795c4a27ed None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4982a0c2-cdb8-4822-ac42-dc795c4a27ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:53:33.200634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:33.201765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:33.202771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:33.202771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:33.202771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:53:33.202771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:53:33.202771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:53:33.202771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:53:33.202771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:53:33.202771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:53:33.202771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 07:53:33.202771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:33.205246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 608/1212] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:33.211646 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26c13a-11d7-4761-8a68-cdd94ff04934 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:33.345398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26c13a-11d7-4761-8a68-cdd94ff04934 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:33.398371 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9f9eb-379b-479f-abb7-f0dc52819a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:33.399096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9f9eb-379b-479f-abb7-f0dc52819a0d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:33.399434 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9f9eb-379b-479f-abb7-f0dc52819a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:33.399800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9f9eb-379b-479f-abb7-f0dc52819a0d None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:33.400715 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9f9eb-379b-479f-abb7-f0dc52819a0d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:33.401109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9f9eb-379b-479f-abb7-f0dc52819a0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:33.402971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9f9eb-379b-479f-abb7-f0dc52819a0d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:33.427948 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 605/1213] 10.222.0.43 () {56 vars in 1143 bytes} [Mon Jul 4 07:53:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:33.434417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982a0c2-cdb8-4822-ac42-dc795c4a27ed None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:33.558482 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982a0c2-cdb8-4822-ac42-dc795c4a27ed None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:33.608705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb1c04-5ebb-4dfd-b520-3d224c20c314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:33.609666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb1c04-5ebb-4dfd-b520-3d224c20c314 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:33.610124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb1c04-5ebb-4dfd-b520-3d224c20c314 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:33.610643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb1c04-5ebb-4dfd-b520-3d224c20c314 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:33.661522 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:33.724564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb1c04-5ebb-4dfd-b520-3d224c20c314 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:33.726552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb1c04-5ebb-4dfd-b520-3d224c20c314 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:33.883351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 609/1214] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:53:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 07:53:33.890039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9f9eb-379b-479f-abb7-f0dc52819a0d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:33.914763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:34.065197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9f9eb-379b-479f-abb7-f0dc52819a0d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:34.111918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb16099-a00f-4f3d-a459-5f51fe36f9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:34.114083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb16099-a00f-4f3d-a459-5f51fe36f9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:34.114453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb16099-a00f-4f3d-a459-5f51fe36f9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:34.115279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb16099-a00f-4f3d-a459-5f51fe36f9b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:34.130095 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb16099-a00f-4f3d-a459-5f51fe36f9b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:34.132237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb16099-a00f-4f3d-a459-5f51fe36f9b6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0bb16099-a00f-4f3d-a459-5f51fe36f9b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:34.152486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:34.154619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:34.156162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:34.156162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:34.156162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:34.156162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:34.156162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 606/1215] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:34.161736 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb1c04-5ebb-4dfd-b520-3d224c20c314 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:34.245255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:34.318086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb1c04-5ebb-4dfd-b520-3d224c20c314 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:34.362568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce58e79-4a08-4911-9f12-0fbb1fcf0707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:34.363280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce58e79-4a08-4911-9f12-0fbb1fcf0707 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:34.363693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce58e79-4a08-4911-9f12-0fbb1fcf0707 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:34.363973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce58e79-4a08-4911-9f12-0fbb1fcf0707 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:34.364813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce58e79-4a08-4911-9f12-0fbb1fcf0707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:34.365133 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce58e79-4a08-4911-9f12-0fbb1fcf0707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:34.366974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce58e79-4a08-4911-9f12-0fbb1fcf0707 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:34.381134 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 610/1216] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:34.388289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb16099-a00f-4f3d-a459-5f51fe36f9b6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:34.486154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:34.527233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb16099-a00f-4f3d-a459-5f51fe36f9b6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:34.570989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbd75e-017b-40b9-9a86-c555daf4a725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:34.571845 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbd75e-017b-40b9-9a86-c555daf4a725 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:34.572134 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbd75e-017b-40b9-9a86-c555daf4a725 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:34.572406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbd75e-017b-40b9-9a86-c555daf4a725 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:34.584038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbd75e-017b-40b9-9a86-c555daf4a725 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:34.586014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbd75e-017b-40b9-9a86-c555daf4a725 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c0bbd75e-017b-40b9-9a86-c555daf4a725 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:53:34.607163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:34.608431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:34.609368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:53:34.609368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:53:34.609368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:53:34.609368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:53:34.609368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:53:34.609368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:53:34.609368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:53:34.609368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:34.613387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 607/1217] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:53:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:34.616470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce58e79-4a08-4911-9f12-0fbb1fcf0707 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:34.739934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce58e79-4a08-4911-9f12-0fbb1fcf0707 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:34.767271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:34.784191 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef45b6-7e85-4b6f-b4d6-7ff8b4270a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:34.784875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef45b6-7e85-4b6f-b4d6-7ff8b4270a56 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:34.785172 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef45b6-7e85-4b6f-b4d6-7ff8b4270a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:34.785484 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef45b6-7e85-4b6f-b4d6-7ff8b4270a56 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:34.786630 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef45b6-7e85-4b6f-b4d6-7ff8b4270a56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:34.786949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef45b6-7e85-4b6f-b4d6-7ff8b4270a56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:34.788536 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef45b6-7e85-4b6f-b4d6-7ff8b4270a56 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:34.804585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 611/1218] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:53:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:34.812454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbd75e-017b-40b9-9a86-c555daf4a725 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:34.947751 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbd75e-017b-40b9-9a86-c555daf4a725 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:35.000749 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:35.012213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9baa0b-25d3-4a9c-9509-24cd3953a361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:35.013303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9baa0b-25d3-4a9c-9509-24cd3953a361 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:35.013582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9baa0b-25d3-4a9c-9509-24cd3953a361 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:35.013933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9baa0b-25d3-4a9c-9509-24cd3953a361 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:35.014986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9baa0b-25d3-4a9c-9509-24cd3953a361 None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=f57dccfe9f5f4bc397f4c470fbf047dc) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:35.015493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9baa0b-25d3-4a9c-9509-24cd3953a361 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:35.017377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9baa0b-25d3-4a9c-9509-24cd3953a361 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:35.029687 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 608/1219] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:53:34 2022] GET /identity/v3/role_assignments?user.id=f57dccfe9f5f4bc397f4c470fbf047dc&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:53:37.299541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff528c-0b85-4f0c-a8b1-c511deeff164 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:37.300124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff528c-0b85-4f0c-a8b1-c511deeff164 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:37.300124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff528c-0b85-4f0c-a8b1-c511deeff164 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:37.302364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 612/1220] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:53:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:53:37.309471 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f9673-8336-4dbd-be44-6a9b1bb981bd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:37.310391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f9673-8336-4dbd-be44-6a9b1bb981bd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:37.310547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f9673-8336-4dbd-be44-6a9b1bb981bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:37.335332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bf4f9673-8336-4dbd-be44-6a9b1bb981bd None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:53:37.337196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bf4f9673-8336-4dbd-be44-6a9b1bb981bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:53:37.423765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 609/1221] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:53:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:53:37.448159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff528c-0b85-4f0c-a8b1-c511deeff164 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:37.578379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff528c-0b85-4f0c-a8b1-c511deeff164 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:37.619156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555e0ec-3b85-4335-9b6b-3ac394e7b474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:37.620040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555e0ec-3b85-4335-9b6b-3ac394e7b474 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:37.620324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555e0ec-3b85-4335-9b6b-3ac394e7b474 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:37.620604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555e0ec-3b85-4335-9b6b-3ac394e7b474 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:37.639559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555e0ec-3b85-4335-9b6b-3ac394e7b474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b555e0ec-3b85-4335-9b6b-3ac394e7b474 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:53:37.648075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:37.649529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:37.651183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 613/1222] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:53:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:53:37.657042 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f9673-8336-4dbd-be44-6a9b1bb981bd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:37.793972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f9673-8336-4dbd-be44-6a9b1bb981bd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:37.829215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:37.843609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056f229-7614-48cf-b043-f81eee18a536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:37.844728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056f229-7614-48cf-b043-f81eee18a536 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:37.845188 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056f229-7614-48cf-b043-f81eee18a536 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:37.845688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056f229-7614-48cf-b043-f81eee18a536 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:37.846885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056f229-7614-48cf-b043-f81eee18a536 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:37.847203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056f229-7614-48cf-b043-f81eee18a536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:37.849059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056f229-7614-48cf-b043-f81eee18a536 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:37.859194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 610/1223] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:53:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:37.868757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555e0ec-3b85-4335-9b6b-3ac394e7b474 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:37.932664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:38.015791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555e0ec-3b85-4335-9b6b-3ac394e7b474 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:38.057849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44095f65-fa57-4111-9d99-76d34b134df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:38.058615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44095f65-fa57-4111-9d99-76d34b134df4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:38.058894 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44095f65-fa57-4111-9d99-76d34b134df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:38.059167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44095f65-fa57-4111-9d99-76d34b134df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:38.071339 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44095f65-fa57-4111-9d99-76d34b134df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:38.072841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44095f65-fa57-4111-9d99-76d34b134df4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-44095f65-fa57-4111-9d99-76d34b134df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:38.091365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:38.092617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:38.093604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:38.093604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:38.093604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:38.093604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:38.093604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 614/1224] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:38.099280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056f229-7614-48cf-b043-f81eee18a536 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:38.222068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056f229-7614-48cf-b043-f81eee18a536 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:38.270579 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c686815-b8fa-4b6b-af13-aa45e3be40da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:38.271268 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c686815-b8fa-4b6b-af13-aa45e3be40da None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:38.271624 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c686815-b8fa-4b6b-af13-aa45e3be40da None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:38.271974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c686815-b8fa-4b6b-af13-aa45e3be40da None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:38.272708 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c686815-b8fa-4b6b-af13-aa45e3be40da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:38.273011 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c686815-b8fa-4b6b-af13-aa45e3be40da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:38.274604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c686815-b8fa-4b6b-af13-aa45e3be40da None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:38.288973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 611/1225] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:53:38.295338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44095f65-fa57-4111-9d99-76d34b134df4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:38.411425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:38.419602 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44095f65-fa57-4111-9d99-76d34b134df4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:38.459126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c820fe5-411b-4726-b059-9b2b49df704a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:38.459847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c820fe5-411b-4726-b059-9b2b49df704a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:38.460066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c820fe5-411b-4726-b059-9b2b49df704a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:38.460463 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c820fe5-411b-4726-b059-9b2b49df704a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:38.470395 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c820fe5-411b-4726-b059-9b2b49df704a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:38.472033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c820fe5-411b-4726-b059-9b2b49df704a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6c820fe5-411b-4726-b059-9b2b49df704a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:53:38.487126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:38.488571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:38.490035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:38.490035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:38.490035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:53:38.490035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:53:38.490035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:53:38.490035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:53:38.490035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:53:38.490035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:53:38.490035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 07:53:38.490035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:38.490035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 615/1226] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:38.494625 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c686815-b8fa-4b6b-af13-aa45e3be40da None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:38.558396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:38.639520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c686815-b8fa-4b6b-af13-aa45e3be40da None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:38.678285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2d55c-6ece-4584-934a-89fa79d73841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:38.678861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2d55c-6ece-4584-934a-89fa79d73841 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:38.679140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2d55c-6ece-4584-934a-89fa79d73841 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:38.679407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2d55c-6ece-4584-934a-89fa79d73841 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:38.680220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2d55c-6ece-4584-934a-89fa79d73841 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:38.680525 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2d55c-6ece-4584-934a-89fa79d73841 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:38.682005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2d55c-6ece-4584-934a-89fa79d73841 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:38.701087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 612/1227] 10.222.0.43 () {56 vars in 1143 bytes} [Mon Jul 4 07:53:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:38.707777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c820fe5-411b-4726-b059-9b2b49df704a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:38.820157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c820fe5-411b-4726-b059-9b2b49df704a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:38.856194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686371c2-f7cd-4c56-b618-d6f673e27620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:38.857103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686371c2-f7cd-4c56-b618-d6f673e27620 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:38.857389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686371c2-f7cd-4c56-b618-d6f673e27620 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:38.857689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686371c2-f7cd-4c56-b618-d6f673e27620 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:38.869351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686371c2-f7cd-4c56-b618-d6f673e27620 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:38.870850 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686371c2-f7cd-4c56-b618-d6f673e27620 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-686371c2-f7cd-4c56-b618-d6f673e27620 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:38.890142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:38.891283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:38.892213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:38.892213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:38.892213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:38.892213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:38.892213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 616/1228] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:38.897664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2d55c-6ece-4584-934a-89fa79d73841 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:39.048089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2d55c-6ece-4584-934a-89fa79d73841 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:39.050774 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:39.088569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47ec06-4a3c-4d38-9933-f5d224b8635d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:39.089273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47ec06-4a3c-4d38-9933-f5d224b8635d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:39.089563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47ec06-4a3c-4d38-9933-f5d224b8635d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:39.089884 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47ec06-4a3c-4d38-9933-f5d224b8635d None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:39.090617 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47ec06-4a3c-4d38-9933-f5d224b8635d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:39.090931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47ec06-4a3c-4d38-9933-f5d224b8635d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:39.092334 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47ec06-4a3c-4d38-9933-f5d224b8635d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:39.111415 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 613/1229] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:39.114042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686371c2-f7cd-4c56-b618-d6f673e27620 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:39.129804 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:39.250681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686371c2-f7cd-4c56-b618-d6f673e27620 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:39.313837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b11ac-2378-4599-8948-4801a166b002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:39.315127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b11ac-2378-4599-8948-4801a166b002 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:39.315577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b11ac-2378-4599-8948-4801a166b002 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:39.316057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b11ac-2378-4599-8948-4801a166b002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:39.328955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b11ac-2378-4599-8948-4801a166b002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:39.331653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b11ac-2378-4599-8948-4801a166b002 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7d1b11ac-2378-4599-8948-4801a166b002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:53:39.349209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:39.355270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:39.357489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:53:39.357489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:53:39.357489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:53:39.357489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:53:39.357489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:53:39.357489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:53:39.357489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:53:39.357489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:39.357489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 617/1230] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:53:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:39.358503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47ec06-4a3c-4d38-9933-f5d224b8635d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:39.489713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47ec06-4a3c-4d38-9933-f5d224b8635d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:39.534724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03f9ed-e2a9-438b-8efc-fcb34f11ea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:39.535572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03f9ed-e2a9-438b-8efc-fcb34f11ea50 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:39.535833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03f9ed-e2a9-438b-8efc-fcb34f11ea50 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:39.536315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03f9ed-e2a9-438b-8efc-fcb34f11ea50 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:39.537236 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03f9ed-e2a9-438b-8efc-fcb34f11ea50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:39.537546 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03f9ed-e2a9-438b-8efc-fcb34f11ea50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:39.539053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03f9ed-e2a9-438b-8efc-fcb34f11ea50 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:39.553899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 614/1231] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:53:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:39.561642 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b11ac-2378-4599-8948-4801a166b002 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:39.647617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:39.694970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b11ac-2378-4599-8948-4801a166b002 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:39.752974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ce8d8-fb08-4be2-a922-838dab976bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:39.754973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ce8d8-fb08-4be2-a922-838dab976bee None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:39.755159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ce8d8-fb08-4be2-a922-838dab976bee None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:39.755886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ce8d8-fb08-4be2-a922-838dab976bee None admin] PATH_INFO: `/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/f57dccfe9f5f4bc397f4c470fbf047dc/roles/fd54d351b4ac49d7be6d0690309c00e7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:39.775794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ce8d8-fb08-4be2-a922-838dab976bee None admin] RBAC: Authorizing `identity:create_grant(project_id=1f2d85c9e3d14bdabfb63fe505d83714, user_id=f57dccfe9f5f4bc397f4c470fbf047dc, role_id=fd54d351b4ac49d7be6d0690309c00e7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:39.777283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ce8d8-fb08-4be2-a922-838dab976bee None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:39.790315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 618/1232] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:53:39 2022] PUT /identity/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/f57dccfe9f5f4bc397f4c470fbf047dc/roles/fd54d351b4ac49d7be6d0690309c00e7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 07:53:42.008826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df14ab-1cec-46c7-ae5a-14f696cac33a None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:42.009426 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df14ab-1cec-46c7-ae5a-14f696cac33a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:42.009535 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df14ab-1cec-46c7-ae5a-14f696cac33a None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:42.011590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 615/1233] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:53:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:53:42.017702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5353e6e-9c57-404d-bebe-f83dc47a264a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:42.018245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5353e6e-9c57-404d-bebe-f83dc47a264a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:42.018245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5353e6e-9c57-404d-bebe-f83dc47a264a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:42.036667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f5353e6e-9c57-404d-bebe-f83dc47a264a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:53:42.038361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f5353e6e-9c57-404d-bebe-f83dc47a264a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:53:42.116426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 619/1234] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:53:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:53:42.143723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df14ab-1cec-46c7-ae5a-14f696cac33a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:42.172617 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:42.268880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df14ab-1cec-46c7-ae5a-14f696cac33a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:42.312235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fef792-c290-41de-bc79-73f017e1a097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:42.313079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fef792-c290-41de-bc79-73f017e1a097 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:42.313364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fef792-c290-41de-bc79-73f017e1a097 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:42.313648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fef792-c290-41de-bc79-73f017e1a097 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:42.324273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fef792-c290-41de-bc79-73f017e1a097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-22fef792-c290-41de-bc79-73f017e1a097 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:53:42.332002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:42.333388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:42.334714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 616/1235] 10.222.0.43 () {56 vars in 1102 bytes} [Mon Jul 4 07:53:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:42.339529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5353e6e-9c57-404d-bebe-f83dc47a264a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:42.388419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:42.451676 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5353e6e-9c57-404d-bebe-f83dc47a264a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:42.493565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9add7-c670-47b8-bbdd-59e51f2355a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:42.494379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9add7-c670-47b8-bbdd-59e51f2355a5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:42.494660 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9add7-c670-47b8-bbdd-59e51f2355a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:42.494936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9add7-c670-47b8-bbdd-59e51f2355a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:42.495695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9add7-c670-47b8-bbdd-59e51f2355a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:42.495999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9add7-c670-47b8-bbdd-59e51f2355a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:42.497392 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9add7-c670-47b8-bbdd-59e51f2355a5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:42.507498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 620/1236] 10.222.0.43 () {56 vars in 1103 bytes} [Mon Jul 4 07:53:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:42.513483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fef792-c290-41de-bc79-73f017e1a097 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:42.608257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:42.626803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fef792-c290-41de-bc79-73f017e1a097 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:42.666747 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f4e96-b8c3-4463-b9e4-34ffc8209ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:42.667595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f4e96-b8c3-4463-b9e4-34ffc8209ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:42.667883 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f4e96-b8c3-4463-b9e4-34ffc8209ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:42.668157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f4e96-b8c3-4463-b9e4-34ffc8209ec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:42.681203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f4e96-b8c3-4463-b9e4-34ffc8209ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:42.682730 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f4e96-b8c3-4463-b9e4-34ffc8209ec1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8b1f4e96-b8c3-4463-b9e4-34ffc8209ec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:42.702414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:42.703711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:42.705061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:42.705061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:42.705061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:42.705061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:42.705977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 617/1237] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:42.710139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9add7-c670-47b8-bbdd-59e51f2355a5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:42.827027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9add7-c670-47b8-bbdd-59e51f2355a5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:42.838441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:42.870463 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a828b0-7b9f-4fd1-9f88-cea4295afbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:42.871049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a828b0-7b9f-4fd1-9f88-cea4295afbca None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:42.871330 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a828b0-7b9f-4fd1-9f88-cea4295afbca None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:42.871607 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a828b0-7b9f-4fd1-9f88-cea4295afbca None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:42.872339 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a828b0-7b9f-4fd1-9f88-cea4295afbca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:42.872644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a828b0-7b9f-4fd1-9f88-cea4295afbca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:42.874073 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a828b0-7b9f-4fd1-9f88-cea4295afbca None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:42.886815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 621/1238] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:42.894093 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f4e96-b8c3-4463-b9e4-34ffc8209ec1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:43.004907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f4e96-b8c3-4463-b9e4-34ffc8209ec1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:43.041677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316c54c-4d16-4f6b-8f77-7563ce256d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:43.042503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316c54c-4d16-4f6b-8f77-7563ce256d14 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:43.042786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316c54c-4d16-4f6b-8f77-7563ce256d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:43.043068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316c54c-4d16-4f6b-8f77-7563ce256d14 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:43.046101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:43.052362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316c54c-4d16-4f6b-8f77-7563ce256d14 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:43.053869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316c54c-4d16-4f6b-8f77-7563ce256d14 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b316c54c-4d16-4f6b-8f77-7563ce256d14 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:53:43.069820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:43.076315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:43.077527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:43.077527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:43.077527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:53:43.077527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:53:43.077527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:53:43.077527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:53:43.077527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:53:43.077527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:53:43.077527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 07:53:43.077527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:43.077527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 618/1239] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:43.078296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a828b0-7b9f-4fd1-9f88-cea4295afbca None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:43.190885 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a828b0-7b9f-4fd1-9f88-cea4295afbca None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:43.228000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448041f-d740-4550-b4b5-937c5fd4682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:43.228718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448041f-d740-4550-b4b5-937c5fd4682e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:43.229009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448041f-d740-4550-b4b5-937c5fd4682e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:43.229295 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448041f-d740-4550-b4b5-937c5fd4682e None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:43.230226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448041f-d740-4550-b4b5-937c5fd4682e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:43.230538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448041f-d740-4550-b4b5-937c5fd4682e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:43.232098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448041f-d740-4550-b4b5-937c5fd4682e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:43.252974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 622/1240] 10.222.0.43 () {56 vars in 1143 bytes} [Mon Jul 4 07:53:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:43.259330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316c54c-4d16-4f6b-8f77-7563ce256d14 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:43.377228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316c54c-4d16-4f6b-8f77-7563ce256d14 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:43.413816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a6b61-c76c-410d-b5c3-d49d89b78a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:43.414477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a6b61-c76c-410d-b5c3-d49d89b78a2a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:43.414753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a6b61-c76c-410d-b5c3-d49d89b78a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:43.415127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a6b61-c76c-410d-b5c3-d49d89b78a2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:43.482371 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:43.490589 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a6b61-c76c-410d-b5c3-d49d89b78a2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:43.492382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a6b61-c76c-410d-b5c3-d49d89b78a2a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:43.601047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 619/1241] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:53:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:53:43.606957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448041f-d740-4550-b4b5-937c5fd4682e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:43.636665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:43.717791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448041f-d740-4550-b4b5-937c5fd4682e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:43.762515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323222b-76d4-43ac-99a9-ae2a61c1e0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:43.763257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323222b-76d4-43ac-99a9-ae2a61c1e0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:43.763538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323222b-76d4-43ac-99a9-ae2a61c1e0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:43.763807 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323222b-76d4-43ac-99a9-ae2a61c1e0f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:43.777561 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323222b-76d4-43ac-99a9-ae2a61c1e0f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:43.779359 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323222b-76d4-43ac-99a9-ae2a61c1e0f0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9323222b-76d4-43ac-99a9-ae2a61c1e0f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:43.803443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:43.804643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:43.805832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:43.805832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:43.805832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:43.805832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:43.810070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 623/1242] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:43.811934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a6b61-c76c-410d-b5c3-d49d89b78a2a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:43.953919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:43.969876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a6b61-c76c-410d-b5c3-d49d89b78a2a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:44.019176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917ab08-0adc-4232-bc27-b2ca3b12034c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:44.020023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917ab08-0adc-4232-bc27-b2ca3b12034c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:44.020314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917ab08-0adc-4232-bc27-b2ca3b12034c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:44.020589 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917ab08-0adc-4232-bc27-b2ca3b12034c None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:44.021397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917ab08-0adc-4232-bc27-b2ca3b12034c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:44.021738 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917ab08-0adc-4232-bc27-b2ca3b12034c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:44.023234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917ab08-0adc-4232-bc27-b2ca3b12034c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:44.036148 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 620/1243] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:44.042247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323222b-76d4-43ac-99a9-ae2a61c1e0f0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:44.146351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:44.171481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323222b-76d4-43ac-99a9-ae2a61c1e0f0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:44.213670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b825fc1-92b5-41fe-8825-cdadd2d04abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:44.214523 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b825fc1-92b5-41fe-8825-cdadd2d04abd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:44.214819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b825fc1-92b5-41fe-8825-cdadd2d04abd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:44.215162 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b825fc1-92b5-41fe-8825-cdadd2d04abd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:44.224711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b825fc1-92b5-41fe-8825-cdadd2d04abd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:44.226292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b825fc1-92b5-41fe-8825-cdadd2d04abd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6b825fc1-92b5-41fe-8825-cdadd2d04abd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:53:44.242219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:44.243731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:44.244698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:53:44.244698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:53:44.244698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:53:44.244698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:53:44.244698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:53:44.244698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:53:44.244698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:53:44.244698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:44.244698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 624/1244] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:53:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:53:44.250676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917ab08-0adc-4232-bc27-b2ca3b12034c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:44.394259 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917ab08-0adc-4232-bc27-b2ca3b12034c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:44.448710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d229fe-7355-40b2-a143-cd963e090ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:44.449582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d229fe-7355-40b2-a143-cd963e090ebd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:44.449998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d229fe-7355-40b2-a143-cd963e090ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:44.450389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d229fe-7355-40b2-a143-cd963e090ebd None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:44.451332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d229fe-7355-40b2-a143-cd963e090ebd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:44.451746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d229fe-7355-40b2-a143-cd963e090ebd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:44.454488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d229fe-7355-40b2-a143-cd963e090ebd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:44.469244 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 621/1245] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:53:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:44.476183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b825fc1-92b5-41fe-8825-cdadd2d04abd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:44.618912 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b825fc1-92b5-41fe-8825-cdadd2d04abd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:44.659937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446988e9-6d72-4c00-a102-d7da6a7c1473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:44.660733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446988e9-6d72-4c00-a102-d7da6a7c1473 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:44.660961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446988e9-6d72-4c00-a102-d7da6a7c1473 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:44.661395 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446988e9-6d72-4c00-a102-d7da6a7c1473 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:44.662878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446988e9-6d72-4c00-a102-d7da6a7c1473 None admin] RBAC: Adding query filter params (role.id=fd54d351b4ac49d7be6d0690309c00e7, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=f57dccfe9f5f4bc397f4c470fbf047dc) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:44.663420 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446988e9-6d72-4c00-a102-d7da6a7c1473 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:44.666006 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446988e9-6d72-4c00-a102-d7da6a7c1473 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:44.674678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 625/1246] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:53:44 2022] GET /identity/v3/role_assignments?user.id=f57dccfe9f5f4bc397f4c470fbf047dc&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=fd54d351b4ac49d7be6d0690309c00e7 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:46.821783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccbd58-c8dc-4638-817c-e1187b9c4130 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:46.822393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccbd58-c8dc-4638-817c-e1187b9c4130 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:46.822393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccbd58-c8dc-4638-817c-e1187b9c4130 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:46.823913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 622/1247] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:53:46.829470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac254bc-e41b-4d1a-9e95-07c932ac9237 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:46.830427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac254bc-e41b-4d1a-9e95-07c932ac9237 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:46.830427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac254bc-e41b-4d1a-9e95-07c932ac9237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:46.836309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:46.850653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3ac254bc-e41b-4d1a-9e95-07c932ac9237 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:53:46.851945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3ac254bc-e41b-4d1a-9e95-07c932ac9237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:53:46.944680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 626/1248] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:53:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:53:46.972029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccbd58-c8dc-4638-817c-e1187b9c4130 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:46.991974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:47.091054 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccbd58-c8dc-4638-817c-e1187b9c4130 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:47.132275 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44413ec3-3732-49e3-bfc4-1eaedc314d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:47.133346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44413ec3-3732-49e3-bfc4-1eaedc314d97 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:47.133813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44413ec3-3732-49e3-bfc4-1eaedc314d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:47.134284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44413ec3-3732-49e3-bfc4-1eaedc314d97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:47.147930 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44413ec3-3732-49e3-bfc4-1eaedc314d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-44413ec3-3732-49e3-bfc4-1eaedc314d97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:53:47.155804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:47.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:47.158238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 623/1249] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:53:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:47.164256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac254bc-e41b-4d1a-9e95-07c932ac9237 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:47.292616 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac254bc-e41b-4d1a-9e95-07c932ac9237 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:47.328186 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:47.334408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c6fbd-eaec-4310-8641-b2e9c655dec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:47.335240 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c6fbd-eaec-4310-8641-b2e9c655dec4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:47.335543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c6fbd-eaec-4310-8641-b2e9c655dec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:47.335833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c6fbd-eaec-4310-8641-b2e9c655dec4 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:47.336548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c6fbd-eaec-4310-8641-b2e9c655dec4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:47.336851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c6fbd-eaec-4310-8641-b2e9c655dec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:47.338403 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c6fbd-eaec-4310-8641-b2e9c655dec4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:47.348044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 627/1250] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:53:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:47.354374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44413ec3-3732-49e3-bfc4-1eaedc314d97 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:47.442622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:47.468558 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44413ec3-3732-49e3-bfc4-1eaedc314d97 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:47.504063 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6c36a-1e37-4930-ab98-f6784d026025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:47.504909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6c36a-1e37-4930-ab98-f6784d026025 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:47.505192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6c36a-1e37-4930-ab98-f6784d026025 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:47.505812 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6c36a-1e37-4930-ab98-f6784d026025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:47.517719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6c36a-1e37-4930-ab98-f6784d026025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:47.520111 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6c36a-1e37-4930-ab98-f6784d026025 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-acc6c36a-1e37-4930-ab98-f6784d026025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:47.543073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:47.544183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:47.545160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:47.545160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:47.545160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:47.545160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:47.545160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 624/1251] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:47.552841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c6fbd-eaec-4310-8641-b2e9c655dec4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:47.692595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c6fbd-eaec-4310-8641-b2e9c655dec4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:47.731212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996849d3-8098-4512-af36-b4ed813148cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:47.731860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996849d3-8098-4512-af36-b4ed813148cc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:47.732155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996849d3-8098-4512-af36-b4ed813148cc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:47.732436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996849d3-8098-4512-af36-b4ed813148cc None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:47.733152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996849d3-8098-4512-af36-b4ed813148cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:47.733442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996849d3-8098-4512-af36-b4ed813148cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:47.734869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996849d3-8098-4512-af36-b4ed813148cc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:47.748354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 628/1252] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:53:47.754686 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6c36a-1e37-4930-ab98-f6784d026025 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:47.892529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6c36a-1e37-4930-ab98-f6784d026025 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:47.924617 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:47.935065 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f450695-2bfd-4038-866f-e8123a8e0df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:47.936020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f450695-2bfd-4038-866f-e8123a8e0df8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:47.936303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f450695-2bfd-4038-866f-e8123a8e0df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:47.936578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f450695-2bfd-4038-866f-e8123a8e0df8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:47.947251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f450695-2bfd-4038-866f-e8123a8e0df8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:47.947251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f450695-2bfd-4038-866f-e8123a8e0df8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2f450695-2bfd-4038-866f-e8123a8e0df8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:53:47.964562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:47.965742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:47.966762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:47.966762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:47.966762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:53:47.966762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:53:47.966762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:53:47.966762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:53:47.966762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:53:47.966762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:53:47.966762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 07:53:47.966762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:47.966762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 625/1253] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:47.972281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996849d3-8098-4512-af36-b4ed813148cc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:48.040665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:48.099684 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996849d3-8098-4512-af36-b4ed813148cc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:48.142652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e1769-2528-42bd-bf5f-17948e99bd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:48.143343 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e1769-2528-42bd-bf5f-17948e99bd30 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:48.143632 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e1769-2528-42bd-bf5f-17948e99bd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:48.143980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e1769-2528-42bd-bf5f-17948e99bd30 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:48.144962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e1769-2528-42bd-bf5f-17948e99bd30 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:48.145277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e1769-2528-42bd-bf5f-17948e99bd30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:48.146782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e1769-2528-42bd-bf5f-17948e99bd30 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:48.167094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 629/1254] 10.222.0.43 () {56 vars in 1143 bytes} [Mon Jul 4 07:53:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:48.173335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f450695-2bfd-4038-866f-e8123a8e0df8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:48.304279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f450695-2bfd-4038-866f-e8123a8e0df8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:48.346112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5da90a-8d79-48f5-ad82-ed095049d969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:48.347002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5da90a-8d79-48f5-ad82-ed095049d969 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:48.347437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5da90a-8d79-48f5-ad82-ed095049d969 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:48.348121 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5da90a-8d79-48f5-ad82-ed095049d969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:48.422610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:48.439655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5da90a-8d79-48f5-ad82-ed095049d969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:48.441519 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5da90a-8d79-48f5-ad82-ed095049d969 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:48.560422 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 626/1255] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:53:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:53:48.566615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e1769-2528-42bd-bf5f-17948e99bd30 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:48.709217 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e1769-2528-42bd-bf5f-17948e99bd30 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:48.724202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:48.750208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02999893-945d-44fc-abff-f25690050022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:48.751031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02999893-945d-44fc-abff-f25690050022 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:48.751308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02999893-945d-44fc-abff-f25690050022 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:48.751590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02999893-945d-44fc-abff-f25690050022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:48.763445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02999893-945d-44fc-abff-f25690050022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:48.765403 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02999893-945d-44fc-abff-f25690050022 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-02999893-945d-44fc-abff-f25690050022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:48.788427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:48.789526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:48.790579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:48.790579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:48.790579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:48.790579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:48.790579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 630/1256] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:48.796327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5da90a-8d79-48f5-ad82-ed095049d969 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:48.893394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:48.909124 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5da90a-8d79-48f5-ad82-ed095049d969 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:48.949873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8820fc-e8a1-42a5-a2ca-185da67642a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:48.950547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8820fc-e8a1-42a5-a2ca-185da67642a7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:48.950826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8820fc-e8a1-42a5-a2ca-185da67642a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:48.951095 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8820fc-e8a1-42a5-a2ca-185da67642a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:48.951830 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8820fc-e8a1-42a5-a2ca-185da67642a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:48.952135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8820fc-e8a1-42a5-a2ca-185da67642a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:48.953538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8820fc-e8a1-42a5-a2ca-185da67642a7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:48.966220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 627/1257] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:53:48.972438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02999893-945d-44fc-abff-f25690050022 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:49.102972 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02999893-945d-44fc-abff-f25690050022 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:49.148419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f42cf-a0a2-4a25-ac2e-b7fd66d9e76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:49.149167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f42cf-a0a2-4a25-ac2e-b7fd66d9e76a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:49.149459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f42cf-a0a2-4a25-ac2e-b7fd66d9e76a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:49.149778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f42cf-a0a2-4a25-ac2e-b7fd66d9e76a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:49.158199 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f42cf-a0a2-4a25-ac2e-b7fd66d9e76a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:49.159888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f42cf-a0a2-4a25-ac2e-b7fd66d9e76a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-530f42cf-a0a2-4a25-ac2e-b7fd66d9e76a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:53:49.175474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:49.176594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:49.177515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:53:49.177515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:53:49.177515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:53:49.177515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:53:49.177515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:53:49.177515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:53:49.177515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:53:49.177515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:49.177515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 631/1258] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:53:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:49.183167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8820fc-e8a1-42a5-a2ca-185da67642a7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:49.312150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8820fc-e8a1-42a5-a2ca-185da67642a7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:49.352063 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d872d-6340-4c80-a3ac-af4f7fa0477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:49.352756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d872d-6340-4c80-a3ac-af4f7fa0477a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:49.353081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d872d-6340-4c80-a3ac-af4f7fa0477a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:49.353389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d872d-6340-4c80-a3ac-af4f7fa0477a None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:49.354353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d872d-6340-4c80-a3ac-af4f7fa0477a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:49.354662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d872d-6340-4c80-a3ac-af4f7fa0477a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:49.356258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d872d-6340-4c80-a3ac-af4f7fa0477a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:49.370213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 628/1259] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:53:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:53:49.376744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f42cf-a0a2-4a25-ac2e-b7fd66d9e76a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:49.412664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:49.497373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f42cf-a0a2-4a25-ac2e-b7fd66d9e76a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:49.535331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d0295-09e1-49d5-8b36-ccd1f7dba78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:49.536103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d0295-09e1-49d5-8b36-ccd1f7dba78d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:49.536445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d0295-09e1-49d5-8b36-ccd1f7dba78d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:49.536783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d0295-09e1-49d5-8b36-ccd1f7dba78d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:49.537806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d0295-09e1-49d5-8b36-ccd1f7dba78d None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=f57dccfe9f5f4bc397f4c470fbf047dc) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:49.538224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d0295-09e1-49d5-8b36-ccd1f7dba78d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:49.540117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d0295-09e1-49d5-8b36-ccd1f7dba78d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:49.547475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 632/1260] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:53:49 2022] GET /identity/v3/role_assignments?user.id=f57dccfe9f5f4bc397f4c470fbf047dc&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:51.640335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6628c2-f959-4522-a2ba-161872b40b6e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:51.640957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6628c2-f959-4522-a2ba-161872b40b6e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:51.640957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6628c2-f959-4522-a2ba-161872b40b6e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:51.642292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 629/1261] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:53:51.648387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc650a5-af49-45b5-9485-cbcea98e7766 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:51.648947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc650a5-af49-45b5-9485-cbcea98e7766 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:51.648947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc650a5-af49-45b5-9485-cbcea98e7766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:51.668397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5dc650a5-af49-45b5-9485-cbcea98e7766 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:53:51.669713 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5dc650a5-af49-45b5-9485-cbcea98e7766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:53:51.752554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 633/1262] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:53:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:53:51.776199 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6628c2-f959-4522-a2ba-161872b40b6e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:51.789303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:51.896443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6628c2-f959-4522-a2ba-161872b40b6e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:51.940228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d8b19-b293-4f50-8af7-16617059ac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:51.941038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d8b19-b293-4f50-8af7-16617059ac22 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:51.941360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d8b19-b293-4f50-8af7-16617059ac22 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:51.941709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d8b19-b293-4f50-8af7-16617059ac22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:51.953345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d8b19-b293-4f50-8af7-16617059ac22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ce9d8b19-b293-4f50-8af7-16617059ac22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:53:51.961147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:51.962389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:51.963568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 630/1263] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:53:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:51.971168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc650a5-af49-45b5-9485-cbcea98e7766 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:51.996152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:52.085985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc650a5-af49-45b5-9485-cbcea98e7766 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:52.126400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37ebc9-86e1-4bec-a898-f00602dfa4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:52.127108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37ebc9-86e1-4bec-a898-f00602dfa4ff None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:52.127418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37ebc9-86e1-4bec-a898-f00602dfa4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:52.127695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37ebc9-86e1-4bec-a898-f00602dfa4ff None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:52.128440 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37ebc9-86e1-4bec-a898-f00602dfa4ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:52.129095 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37ebc9-86e1-4bec-a898-f00602dfa4ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:52.131086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37ebc9-86e1-4bec-a898-f00602dfa4ff None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:52.140893 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 634/1264] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:53:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:52.147677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d8b19-b293-4f50-8af7-16617059ac22 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:52.228086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:52.269079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d8b19-b293-4f50-8af7-16617059ac22 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:52.306575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac870c2-d267-488c-9c89-92b462868bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:52.307392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac870c2-d267-488c-9c89-92b462868bdc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:52.307663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac870c2-d267-488c-9c89-92b462868bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:52.307936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac870c2-d267-488c-9c89-92b462868bdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:52.320261 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac870c2-d267-488c-9c89-92b462868bdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:52.321751 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac870c2-d267-488c-9c89-92b462868bdc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5ac870c2-d267-488c-9c89-92b462868bdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:52.339380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:52.340516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:52.341688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:52.341688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:52.341688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:52.341688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:52.341688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 631/1265] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:52.346985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37ebc9-86e1-4bec-a898-f00602dfa4ff None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:52.442569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:52.463899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37ebc9-86e1-4bec-a898-f00602dfa4ff None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:52.501883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37ff2c-08f0-4106-b16c-6d08463714e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:52.502562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37ff2c-08f0-4106-b16c-6d08463714e3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:52.502845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37ff2c-08f0-4106-b16c-6d08463714e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:52.503121 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37ff2c-08f0-4106-b16c-6d08463714e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:52.503903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37ff2c-08f0-4106-b16c-6d08463714e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:52.504892 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37ff2c-08f0-4106-b16c-6d08463714e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:52.506689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37ff2c-08f0-4106-b16c-6d08463714e3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:52.519244 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 635/1266] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:52.526174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac870c2-d267-488c-9c89-92b462868bdc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:52.632986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac870c2-d267-488c-9c89-92b462868bdc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:52.654446 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:52.672805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bc722-a54c-4cb6-a4e6-4e6fe14a1393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:52.673645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bc722-a54c-4cb6-a4e6-4e6fe14a1393 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:52.673925 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bc722-a54c-4cb6-a4e6-4e6fe14a1393 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:52.674194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bc722-a54c-4cb6-a4e6-4e6fe14a1393 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:52.682692 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469bc722-a54c-4cb6-a4e6-4e6fe14a1393 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:52.684265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469bc722-a54c-4cb6-a4e6-4e6fe14a1393 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-469bc722-a54c-4cb6-a4e6-4e6fe14a1393 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:53:52.703013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:52.704194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:52.705175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:52.705175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:52.705175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:53:52.705175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:53:52.705175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:53:52.705175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:53:52.705175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:53:52.705175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:53:52.705175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 07:53:52.705175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:52.705175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 632/1267] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:52.715423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37ff2c-08f0-4106-b16c-6d08463714e3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:52.838079 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37ff2c-08f0-4106-b16c-6d08463714e3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:52.879495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0fbd6-f817-49b3-ab30-c62744b0030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:52.880246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0fbd6-f817-49b3-ab30-c62744b0030b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:52.880876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0fbd6-f817-49b3-ab30-c62744b0030b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:52.881194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0fbd6-f817-49b3-ab30-c62744b0030b None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:52.882085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0fbd6-f817-49b3-ab30-c62744b0030b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:52.882399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0fbd6-f817-49b3-ab30-c62744b0030b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:52.883861 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0fbd6-f817-49b3-ab30-c62744b0030b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:52.893286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:52.902182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 636/1268] 10.222.0.43 () {56 vars in 1143 bytes} [Mon Jul 4 07:53:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:52.908087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bc722-a54c-4cb6-a4e6-4e6fe14a1393 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:53.010710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bc722-a54c-4cb6-a4e6-4e6fe14a1393 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:53.046260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3a024-769a-4564-8341-fd4e4a2947cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:53.047059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3a024-769a-4564-8341-fd4e4a2947cb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:53.047335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3a024-769a-4564-8341-fd4e4a2947cb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:53.047602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3a024-769a-4564-8341-fd4e4a2947cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:53.059575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3a024-769a-4564-8341-fd4e4a2947cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:53.061033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3a024-769a-4564-8341-fd4e4a2947cb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-54f3a024-769a-4564-8341-fd4e4a2947cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:53.078721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:53.079718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:53.080739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:53.080739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:53.080739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:53.080739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:53.080739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 633/1269] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:53:53.088455 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0fbd6-f817-49b3-ab30-c62744b0030b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:53.217192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0fbd6-f817-49b3-ab30-c62744b0030b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:53.259801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a58a4b-4a1b-405c-a8d1-df2d02455435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:53.260545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a58a4b-4a1b-405c-a8d1-df2d02455435 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:53.260924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a58a4b-4a1b-405c-a8d1-df2d02455435 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:53.261158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a58a4b-4a1b-405c-a8d1-df2d02455435 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:53.262039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a58a4b-4a1b-405c-a8d1-df2d02455435 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:53.262398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a58a4b-4a1b-405c-a8d1-df2d02455435 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:53.263903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a58a4b-4a1b-405c-a8d1-df2d02455435 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:53.278656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 637/1270] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:53.286216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3a024-769a-4564-8341-fd4e4a2947cb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:53.321441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:53.392676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3a024-769a-4564-8341-fd4e4a2947cb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:53.427373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fac30-9269-4dea-8d43-a23bf79e6334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:53.428164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fac30-9269-4dea-8d43-a23bf79e6334 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:53.428445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fac30-9269-4dea-8d43-a23bf79e6334 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:53.428717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fac30-9269-4dea-8d43-a23bf79e6334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:53.435673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fac30-9269-4dea-8d43-a23bf79e6334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:53.437154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fac30-9269-4dea-8d43-a23bf79e6334 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7f4fac30-9269-4dea-8d43-a23bf79e6334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:53:53.451542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:53.452600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:53.454002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:53:53.454002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:53:53.454002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:53:53.454002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:53:53.454002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:53:53.454002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:53:53.454002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:53:53.454002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:53.454002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 634/1271] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:53:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:53.459055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a58a4b-4a1b-405c-a8d1-df2d02455435 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:53.523910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:53.568225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a58a4b-4a1b-405c-a8d1-df2d02455435 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:53.614128 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a6b11-f07f-43e3-9604-f57e29134b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:53.614811 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a6b11-f07f-43e3-9604-f57e29134b4a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:53.615097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a6b11-f07f-43e3-9604-f57e29134b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:53.615375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a6b11-f07f-43e3-9604-f57e29134b4a None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:53.616177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a6b11-f07f-43e3-9604-f57e29134b4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:53.616492 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a6b11-f07f-43e3-9604-f57e29134b4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:53.618337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a6b11-f07f-43e3-9604-f57e29134b4a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:53.633133 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 638/1272] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:53:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:53.639606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fac30-9269-4dea-8d43-a23bf79e6334 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:53.739911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:53.749433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fac30-9269-4dea-8d43-a23bf79e6334 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:53.786131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe295b-98ea-45f9-a25e-b4471d913850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:53.786944 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe295b-98ea-45f9-a25e-b4471d913850 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:53.787226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe295b-98ea-45f9-a25e-b4471d913850 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:53.787501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe295b-98ea-45f9-a25e-b4471d913850 None admin] PATH_INFO: `/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/f57dccfe9f5f4bc397f4c470fbf047dc/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:53.789513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe295b-98ea-45f9-a25e-b4471d913850 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f2d85c9e3d14bdabfb63fe505d83714, user_id=f57dccfe9f5f4bc397f4c470fbf047dc, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:53.791018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe295b-98ea-45f9-a25e-b4471d913850 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:53.805938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 635/1273] 10.222.0.43 () {58 vars in 1439 bytes} [Mon Jul 4 07:53:53 2022] PUT /identity/v3/projects/1f2d85c9e3d14bdabfb63fe505d83714/users/f57dccfe9f5f4bc397f4c470fbf047dc/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:53:55.801569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8058fa-1742-421d-b858-ebd7f2492e2a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:55.802227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8058fa-1742-421d-b858-ebd7f2492e2a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:55.802227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8058fa-1742-421d-b858-ebd7f2492e2a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:55.803573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 639/1274] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:53:55.808858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a292d6b-9e18-4d8e-8fb1-21a8301fd2ca None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:55.809321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a292d6b-9e18-4d8e-8fb1-21a8301fd2ca None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:55.809439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a292d6b-9e18-4d8e-8fb1-21a8301fd2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:55.827711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3a292d6b-9e18-4d8e-8fb1-21a8301fd2ca None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:53:55.828967 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3a292d6b-9e18-4d8e-8fb1-21a8301fd2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:53:55.898262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 636/1275] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:53:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:53:55.922074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8058fa-1742-421d-b858-ebd7f2492e2a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:56.028707 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8058fa-1742-421d-b858-ebd7f2492e2a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:56.047211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:56.064871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340039c-1d0f-43a7-b948-f6d0aca53874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:56.065587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340039c-1d0f-43a7-b948-f6d0aca53874 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:56.065903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340039c-1d0f-43a7-b948-f6d0aca53874 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:56.066170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340039c-1d0f-43a7-b948-f6d0aca53874 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:56.076390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340039c-1d0f-43a7-b948-f6d0aca53874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c340039c-1d0f-43a7-b948-f6d0aca53874 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 07:53:56.084078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:56.085168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:56.086257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:56.086257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:56.086257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:56.086257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:56.086257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 07:53:56.086257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 07:53:56.086257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 07:53:56.086257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 07:53:56.086257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 07:53:56.086257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 07:53:56.086257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 07:53:56.086257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:56.089824 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 640/1276] 10.222.0.43 () {56 vars in 1096 bytes} [Mon Jul 4 07:53:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:56.091386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a292d6b-9e18-4d8e-8fb1-21a8301fd2ca None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:56.179410 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:56.201901 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a292d6b-9e18-4d8e-8fb1-21a8301fd2ca None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:56.238957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb58b3-c1c9-46f9-a4cd-ceaa9d3e749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:56.239779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb58b3-c1c9-46f9-a4cd-ceaa9d3e749d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:56.240205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb58b3-c1c9-46f9-a4cd-ceaa9d3e749d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:56.240428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb58b3-c1c9-46f9-a4cd-ceaa9d3e749d None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:56.241350 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb58b3-c1c9-46f9-a4cd-ceaa9d3e749d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:56.241753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb58b3-c1c9-46f9-a4cd-ceaa9d3e749d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:56.243369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb58b3-c1c9-46f9-a4cd-ceaa9d3e749d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:56.253093 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 637/1277] 10.222.0.43 () {56 vars in 1099 bytes} [Mon Jul 4 07:53:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:56.259472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340039c-1d0f-43a7-b948-f6d0aca53874 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:56.368547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340039c-1d0f-43a7-b948-f6d0aca53874 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:56.404059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b5bd5-0e02-4156-9a22-4759e2b525c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:56.404855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b5bd5-0e02-4156-9a22-4759e2b525c9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:56.405137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b5bd5-0e02-4156-9a22-4759e2b525c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:56.405413 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b5bd5-0e02-4156-9a22-4759e2b525c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:56.416740 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b5bd5-0e02-4156-9a22-4759e2b525c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:56.418237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b5bd5-0e02-4156-9a22-4759e2b525c9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ca9b5bd5-0e02-4156-9a22-4759e2b525c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:56.435826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:56.438604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:56.439671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:56.439671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:56.439671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:56.439671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:56.439671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 641/1278] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:56.443778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb58b3-c1c9-46f9-a4cd-ceaa9d3e749d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:56.548965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb58b3-c1c9-46f9-a4cd-ceaa9d3e749d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:56.584237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835dc4a7-fcab-4b0b-be81-43b25c80d0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:56.584905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835dc4a7-fcab-4b0b-be81-43b25c80d0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:56.585192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835dc4a7-fcab-4b0b-be81-43b25c80d0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:56.585466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835dc4a7-fcab-4b0b-be81-43b25c80d0e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:56.586226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835dc4a7-fcab-4b0b-be81-43b25c80d0e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:56.586534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835dc4a7-fcab-4b0b-be81-43b25c80d0e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:56.587923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835dc4a7-fcab-4b0b-be81-43b25c80d0e7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:56.600388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 638/1279] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:53:56.606388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b5bd5-0e02-4156-9a22-4759e2b525c9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:56.640725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:56.713864 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b5bd5-0e02-4156-9a22-4759e2b525c9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:56.749289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70c70b-779b-47ad-b894-742e2cd12ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:56.750131 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70c70b-779b-47ad-b894-742e2cd12ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:56.750417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70c70b-779b-47ad-b894-742e2cd12ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:56.750692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70c70b-779b-47ad-b894-742e2cd12ad5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:56.759207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70c70b-779b-47ad-b894-742e2cd12ad5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:56.760658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70c70b-779b-47ad-b894-742e2cd12ad5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0c70c70b-779b-47ad-b894-742e2cd12ad5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 07:53:56.775407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:56.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:56.777992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:56.777992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:56.777992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 07:53:56.777992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 07:53:56.777992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 07:53:56.777992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 07:53:56.777992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 07:53:56.777992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 07:53:56.777992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 07:53:56.777992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:53:56.777992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 642/1280] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:56.782662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835dc4a7-fcab-4b0b-be81-43b25c80d0e7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:56.786907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:56.891408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835dc4a7-fcab-4b0b-be81-43b25c80d0e7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:56.931912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86697eed-12dd-4112-b2d5-69ef84371f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:56.932623 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86697eed-12dd-4112-b2d5-69ef84371f6b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:56.932905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86697eed-12dd-4112-b2d5-69ef84371f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:56.933178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86697eed-12dd-4112-b2d5-69ef84371f6b None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:56.934087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86697eed-12dd-4112-b2d5-69ef84371f6b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:56.934405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86697eed-12dd-4112-b2d5-69ef84371f6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:56.935908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86697eed-12dd-4112-b2d5-69ef84371f6b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:56.956514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 639/1281] 10.222.0.43 () {56 vars in 1143 bytes} [Mon Jul 4 07:53:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:56.963498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70c70b-779b-47ad-b894-742e2cd12ad5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:57.060109 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:57.070472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70c70b-779b-47ad-b894-742e2cd12ad5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:57.105110 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64b1fb-a234-465b-b6af-eefdd8380970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:57.105809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64b1fb-a234-465b-b6af-eefdd8380970 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:57.106093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64b1fb-a234-465b-b6af-eefdd8380970 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:57.106349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64b1fb-a234-465b-b6af-eefdd8380970 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:57.180316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64b1fb-a234-465b-b6af-eefdd8380970 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:57.181720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64b1fb-a234-465b-b6af-eefdd8380970 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:57.283331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:57.295001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 643/1282] 10.222.0.43 () {58 vars in 1282 bytes} [Mon Jul 4 07:53:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 07:53:57.301588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86697eed-12dd-4112-b2d5-69ef84371f6b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:57.366938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:57.406931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86697eed-12dd-4112-b2d5-69ef84371f6b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:57.442543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180a6f3-8a28-43de-a4e1-6117e4b8e8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:57.443271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180a6f3-8a28-43de-a4e1-6117e4b8e8ce None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:57.443553 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180a6f3-8a28-43de-a4e1-6117e4b8e8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:57.443824 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180a6f3-8a28-43de-a4e1-6117e4b8e8ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:57.456536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180a6f3-8a28-43de-a4e1-6117e4b8e8ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:57.458191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180a6f3-8a28-43de-a4e1-6117e4b8e8ce None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c180a6f3-8a28-43de-a4e1-6117e4b8e8ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 07:53:57.476757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:57.477920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:57.479265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 07:53:57.479265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 07:53:57.479265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 07:53:57.479265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:57.479265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 640/1283] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:57.484026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64b1fb-a234-465b-b6af-eefdd8380970 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:57.590243 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64b1fb-a234-465b-b6af-eefdd8380970 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:57.627038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204bee4-8673-463e-801e-e40cd533bd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:57.627719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204bee4-8673-463e-801e-e40cd533bd36 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:57.628003 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204bee4-8673-463e-801e-e40cd533bd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:57.628276 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204bee4-8673-463e-801e-e40cd533bd36 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:57.629290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204bee4-8673-463e-801e-e40cd533bd36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:57.629290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204bee4-8673-463e-801e-e40cd533bd36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:57.631104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204bee4-8673-463e-801e-e40cd533bd36 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:57.643373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 644/1284] 10.222.0.43 () {56 vars in 1109 bytes} [Mon Jul 4 07:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:57.649433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180a6f3-8a28-43de-a4e1-6117e4b8e8ce None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:57.754731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180a6f3-8a28-43de-a4e1-6117e4b8e8ce None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:57.769658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:57.792849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae9f59-38b0-40cc-b1ea-201a79f627e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:57.793669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae9f59-38b0-40cc-b1ea-201a79f627e3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:57.793970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae9f59-38b0-40cc-b1ea-201a79f627e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:57.794245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae9f59-38b0-40cc-b1ea-201a79f627e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:57.801429 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae9f59-38b0-40cc-b1ea-201a79f627e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:57.802986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae9f59-38b0-40cc-b1ea-201a79f627e3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-97ae9f59-38b0-40cc-b1ea-201a79f627e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 07:53:57.816961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:53:57.817981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:53:57.819085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 07:53:57.819085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 07:53:57.819085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 07:53:57.819085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 07:53:57.819085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 07:53:57.819085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 07:53:57.819085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 07:53:57.819085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Jul 04 07:53:57.821787 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 641/1285] 10.222.0.43 () {56 vars in 1111 bytes} [Mon Jul 4 07:53:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:53:57.824286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204bee4-8673-463e-801e-e40cd533bd36 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:57.881326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:53:57.934335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204bee4-8673-463e-801e-e40cd533bd36 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:57.970109 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab60b6-5278-4642-80c5-68890c4a0e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:57.970793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab60b6-5278-4642-80c5-68890c4a0e10 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:57.971037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab60b6-5278-4642-80c5-68890c4a0e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:57.971315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab60b6-5278-4642-80c5-68890c4a0e10 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:57.972106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab60b6-5278-4642-80c5-68890c4a0e10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:57.972433 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab60b6-5278-4642-80c5-68890c4a0e10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:57.973932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab60b6-5278-4642-80c5-68890c4a0e10 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:57.986274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 645/1286] 10.222.0.43 () {56 vars in 1148 bytes} [Mon Jul 4 07:53:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:53:57.992620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae9f59-38b0-40cc-b1ea-201a79f627e3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:53:58.095698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae9f59-38b0-40cc-b1ea-201a79f627e3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:53:58.133025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817231c-eb8e-40d0-bb05-28997373976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:53:58.133720 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817231c-eb8e-40d0-bb05-28997373976f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:53:58.134000 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817231c-eb8e-40d0-bb05-28997373976f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:53:58.134274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817231c-eb8e-40d0-bb05-28997373976f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:53:58.135106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817231c-eb8e-40d0-bb05-28997373976f None admin] RBAC: Adding query filter params (role.id=e60a0103aa774cff9ee9b873bca64fb6, scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714, user.id=f57dccfe9f5f4bc397f4c470fbf047dc) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:53:58.135415 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817231c-eb8e-40d0-bb05-28997373976f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:53:58.136883 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817231c-eb8e-40d0-bb05-28997373976f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:53:58.143792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 642/1287] 10.222.0.43 () {56 vars in 1374 bytes} [Mon Jul 4 07:53:57 2022] GET /identity/v3/role_assignments?user.id=f57dccfe9f5f4bc397f4c470fbf047dc&scope.project.id=1f2d85c9e3d14bdabfb63fe505d83714&role.id=e60a0103aa774cff9ee9b873bca64fb6 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:54:00.285481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06e316-f1ad-497f-8e3d-b5bd3192bbbf None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:00.286193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06e316-f1ad-497f-8e3d-b5bd3192bbbf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:00.286193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06e316-f1ad-497f-8e3d-b5bd3192bbbf None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:00.288328 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 646/1288] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:54:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:54:00.296403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4033f-ba10-4814-8642-993de94c66a0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:00.296964 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4033f-ba10-4814-8642-993de94c66a0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:00.297471 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4033f-ba10-4814-8642-993de94c66a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:00.318454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-15f4033f-ba10-4814-8642-993de94c66a0 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:54:00.319706 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-15f4033f-ba10-4814-8642-993de94c66a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:54:00.351415 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:54:00.390736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 643/1289] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:54:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:54:00.413670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06e316-f1ad-497f-8e3d-b5bd3192bbbf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:54:00.522870 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06e316-f1ad-497f-8e3d-b5bd3192bbbf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:54:00.534319 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:54:00.568131 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452fcd7-28de-4744-a043-e2de27cbed7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:54:00.568947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452fcd7-28de-4744-a043-e2de27cbed7d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:00.569235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452fcd7-28de-4744-a043-e2de27cbed7d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:00.569514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452fcd7-28de-4744-a043-e2de27cbed7d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:00.570216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452fcd7-28de-4744-a043-e2de27cbed7d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:54:00.571614 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452fcd7-28de-4744-a043-e2de27cbed7d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d452fcd7-28de-4744-a043-e2de27cbed7d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:54:00.584605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:54:00.585975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:54:00.587198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 647/1290] 10.222.0.43 () {56 vars in 1117 bytes} [Mon Jul 4 07:54:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:54:00.593841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4033f-ba10-4814-8642-993de94c66a0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:54:00.704101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4033f-ba10-4814-8642-993de94c66a0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:54:00.739358 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e6ec2-be37-4d64-bafc-8e6522dd8415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:54:00.740023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e6ec2-be37-4d64-bafc-8e6522dd8415 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:00.740305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e6ec2-be37-4d64-bafc-8e6522dd8415 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:00.740697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e6ec2-be37-4d64-bafc-8e6522dd8415 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:00.741444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e6ec2-be37-4d64-bafc-8e6522dd8415 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:54:00.741784 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e6ec2-be37-4d64-bafc-8e6522dd8415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:54:00.743196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e6ec2-be37-4d64-bafc-8e6522dd8415 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:54:00.748068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 644/1291] 10.222.0.43 () {56 vars in 1116 bytes} [Mon Jul 4 07:54:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:54:00.754249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452fcd7-28de-4744-a043-e2de27cbed7d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:54:00.856167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452fcd7-28de-4744-a043-e2de27cbed7d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:54:00.891849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac11638-3e9b-40b6-b79d-94324ace446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:54:00.892506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac11638-3e9b-40b6-b79d-94324ace446e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:00.892802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac11638-3e9b-40b6-b79d-94324ace446e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:00.893074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac11638-3e9b-40b6-b79d-94324ace446e None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:00.893816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac11638-3e9b-40b6-b79d-94324ace446e None admin] RBAC: Adding query filter params (type=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:54:00.894126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac11638-3e9b-40b6-b79d-94324ace446e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:54:00.895746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac11638-3e9b-40b6-b79d-94324ace446e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:54:00.900352 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 648/1292] 10.222.0.43 () {56 vars in 1116 bytes} [Mon Jul 4 07:54:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:54:02.923261 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ad387-4e3c-4a94-a40e-817c2a97cd92 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:02.923876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ad387-4e3c-4a94-a40e-817c2a97cd92 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:02.923876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ad387-4e3c-4a94-a40e-817c2a97cd92 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:02.925209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 645/1293] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:54:02.930642 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc18ba6-5041-40c9-a0fa-82914b34d830 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:02.931097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc18ba6-5041-40c9-a0fa-82914b34d830 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:02.931192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc18ba6-5041-40c9-a0fa-82914b34d830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:02.949471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9cc18ba6-5041-40c9-a0fa-82914b34d830 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:54:02.950719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9cc18ba6-5041-40c9-a0fa-82914b34d830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:54:02.983336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:54:03.025155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 649/1294] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:54:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:54:03.049171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ad387-4e3c-4a94-a40e-817c2a97cd92 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:54:03.088973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:54:03.167819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ad387-4e3c-4a94-a40e-817c2a97cd92 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:54:03.205283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f81b47-e6e1-4d38-9bf1-1607d902da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:54:03.205889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f81b47-e6e1-4d38-9bf1-1607d902da3a None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:03.206170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f81b47-e6e1-4d38-9bf1-1607d902da3a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:03.206441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f81b47-e6e1-4d38-9bf1-1607d902da3a None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:03.207340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f81b47-e6e1-4d38-9bf1-1607d902da3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:54:03.208749 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f81b47-e6e1-4d38-9bf1-1607d902da3a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:54:03.224358 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 646/1295] 10.222.0.43 () {60 vars in 1141 bytes} [Mon Jul 4 07:54:03 2022] POST /identity/v3/services => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:54:05.386302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3091329-a86b-43c1-826e-13c23b511771 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:05.386841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3091329-a86b-43c1-826e-13c23b511771 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:05.386841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3091329-a86b-43c1-826e-13c23b511771 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:05.388128 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 650/1296] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:54:05.393919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88299227-657e-4a76-bad9-1c5dc6e2bd6a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:05.394654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88299227-657e-4a76-bad9-1c5dc6e2bd6a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:05.394654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88299227-657e-4a76-bad9-1c5dc6e2bd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:05.412597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-88299227-657e-4a76-bad9-1c5dc6e2bd6a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:54:05.413858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-88299227-657e-4a76-bad9-1c5dc6e2bd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:54:05.493934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 647/1297] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:54:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:54:05.517604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3091329-a86b-43c1-826e-13c23b511771 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:54:05.635704 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3091329-a86b-43c1-826e-13c23b511771 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:54:05.678331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c5159-ebb9-46b4-855a-043137e72032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:54:05.679062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c5159-ebb9-46b4-855a-043137e72032 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:05.679346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c5159-ebb9-46b4-855a-043137e72032 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:05.679623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c5159-ebb9-46b4-855a-043137e72032 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:05.680288 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c5159-ebb9-46b4-855a-043137e72032 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:54:05.681746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c5159-ebb9-46b4-855a-043137e72032 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c04c5159-ebb9-46b4-855a-043137e72032 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:54:05.694763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:54:05.696167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:54:05.697209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 651/1298] 10.222.0.43 () {56 vars in 1117 bytes} [Mon Jul 4 07:54:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 07:54:05.702373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88299227-657e-4a76-bad9-1c5dc6e2bd6a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:54:05.720809 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:54:05.813753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88299227-657e-4a76-bad9-1c5dc6e2bd6a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:54:05.852055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cdb74-6275-46f1-ac0a-dd684425e8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:54:05.852793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cdb74-6275-46f1-ac0a-dd684425e8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:05.853123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cdb74-6275-46f1-ac0a-dd684425e8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:05.853431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cdb74-6275-46f1-ac0a-dd684425e8f1 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:05.854228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cdb74-6275-46f1-ac0a-dd684425e8f1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:54:05.854582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cdb74-6275-46f1-ac0a-dd684425e8f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:54:05.856023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cdb74-6275-46f1-ac0a-dd684425e8f1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:54:05.861726 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 648/1299] 10.222.0.43 () {56 vars in 1116 bytes} [Mon Jul 4 07:54:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 07:54:05.868100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c5159-ebb9-46b4-855a-043137e72032 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:54:05.902877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:54:05.984757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c5159-ebb9-46b4-855a-043137e72032 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:54:06.021979 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656de9c-8e6b-4350-b28a-00c3a4eeecac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:54:06.022651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656de9c-8e6b-4350-b28a-00c3a4eeecac None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:06.022931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656de9c-8e6b-4350-b28a-00c3a4eeecac None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:06.023203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656de9c-8e6b-4350-b28a-00c3a4eeecac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:06.024000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656de9c-8e6b-4350-b28a-00c3a4eeecac None admin] RBAC: Adding query filter params (interface=public, service_id=4af54b8f505141058790a8080fa0966b, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:54:06.024296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656de9c-8e6b-4350-b28a-00c3a4eeecac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:54:06.025691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656de9c-8e6b-4350-b28a-00c3a4eeecac None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:54:06.031221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 652/1300] 10.222.0.43 () {56 vars in 1251 bytes} [Mon Jul 4 07:54:05 2022] GET /identity/v3/endpoints?service_id=4af54b8f505141058790a8080fa0966b&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:54:06.037648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cdb74-6275-46f1-ac0a-dd684425e8f1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:54:06.125753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:54:06.153638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cdb74-6275-46f1-ac0a-dd684425e8f1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:54:06.193328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9ee7b-f772-48b7-bec5-c2fb38502a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:54:06.194030 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9ee7b-f772-48b7-bec5-c2fb38502a9a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:06.194311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9ee7b-f772-48b7-bec5-c2fb38502a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:06.194583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9ee7b-f772-48b7-bec5-c2fb38502a9a None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:06.195274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9ee7b-f772-48b7-bec5-c2fb38502a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:54:06.196652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9ee7b-f772-48b7-bec5-c2fb38502a9a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:54:06.203947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 649/1301] 10.222.0.43 () {56 vars in 1087 bytes} [Mon Jul 4 07:54:06 2022] GET /identity/v3/services => generated 2191 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 07:54:08.259946 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5406c2-341a-41f0-bdf5-c8c585998059 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:08.260553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5406c2-341a-41f0-bdf5-c8c585998059 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:08.260910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5406c2-341a-41f0-bdf5-c8c585998059 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:08.263398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 653/1302] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:54:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:54:08.269193 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c8322-6521-4783-b958-569857f484cf None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:08.269932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c8322-6521-4783-b958-569857f484cf None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:08.269932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c8322-6521-4783-b958-569857f484cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:08.288007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2c3c8322-6521-4783-b958-569857f484cf None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:54:08.289221 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2c3c8322-6521-4783-b958-569857f484cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:54:08.363964 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 650/1303] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:54:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 07:54:08.389437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5406c2-341a-41f0-bdf5-c8c585998059 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:54:08.541953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:54:08.551445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5406c2-341a-41f0-bdf5-c8c585998059 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:54:08.594800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99960cd1-d149-4208-a508-c06a97a6e8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:54:08.595699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99960cd1-d149-4208-a508-c06a97a6e8ae None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:08.596029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99960cd1-d149-4208-a508-c06a97a6e8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:08.596349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99960cd1-d149-4208-a508-c06a97a6e8ae None admin] PATH_INFO: `/v3/services/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:08.597074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99960cd1-d149-4208-a508-c06a97a6e8ae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:54:08.599024 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99960cd1-d149-4208-a508-c06a97a6e8ae None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-99960cd1-d149-4208-a508-c06a97a6e8ae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 07:54:08.612444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 07:54:08.614533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 04 07:54:08.616173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 654/1304] 10.222.0.43 () {56 vars in 1117 bytes} [Mon Jul 4 07:54:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 07:54:08.621274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c8322-6521-4783-b958-569857f484cf None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:54:08.715353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:54:08.743627 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c8322-6521-4783-b958-569857f484cf None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:54:08.782448 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff8608-12ad-47c0-a9c1-b130b2634298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:54:08.783133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff8608-12ad-47c0-a9c1-b130b2634298 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:08.783421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff8608-12ad-47c0-a9c1-b130b2634298 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:08.783694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff8608-12ad-47c0-a9c1-b130b2634298 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:08.784440 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff8608-12ad-47c0-a9c1-b130b2634298 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:54:08.784780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff8608-12ad-47c0-a9c1-b130b2634298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:54:08.786650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff8608-12ad-47c0-a9c1-b130b2634298 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:54:08.792534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 651/1305] 10.222.0.43 () {56 vars in 1116 bytes} [Mon Jul 4 07:54:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:54:08.798705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99960cd1-d149-4208-a508-c06a97a6e8ae None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:54:08.920014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99960cd1-d149-4208-a508-c06a97a6e8ae None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:54:08.961514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed70ce-480b-4c92-977f-6af1b483e7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:54:08.962283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed70ce-480b-4c92-977f-6af1b483e7c5 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:54:08.962618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed70ce-480b-4c92-977f-6af1b483e7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:54:08.962958 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed70ce-480b-4c92-977f-6af1b483e7c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:54:08.963782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed70ce-480b-4c92-977f-6af1b483e7c5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:54:08.965257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed70ce-480b-4c92-977f-6af1b483e7c5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:54:08.987712 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 655/1306] 10.222.0.43 () {60 vars in 1144 bytes} [Mon Jul 4 07:54:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:54:18.789144 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=30 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:54:18.982769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:17.278787 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132ade3-3e76-433f-90ff-aea154307ffa None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:17.279294 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132ade3-3e76-433f-90ff-aea154307ffa None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:17.279495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132ade3-3e76-433f-90ff-aea154307ffa None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:17.281326 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 652/1307] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:17.287212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813aa7c-1a68-40bd-8de1-e4f6defcbbd9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:17.287655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813aa7c-1a68-40bd-8de1-e4f6defcbbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:17.287848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813aa7c-1a68-40bd-8de1-e4f6defcbbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:17.437013 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4813aa7c-1a68-40bd-8de1-e4f6defcbbd9 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:17.438314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4813aa7c-1a68-40bd-8de1-e4f6defcbbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:17.673855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 656/1308] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:17.731983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243fd92-d023-46f7-98d2-9e2e3d1243bb None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:17.732782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243fd92-d023-46f7-98d2-9e2e3d1243bb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:17.733344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243fd92-d023-46f7-98d2-9e2e3d1243bb None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:17.736764 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 653/1309] 10.222.0.43 () {54 vars in 964 bytes} [Mon Jul 4 07:55:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:55:17.746827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd444ea-061f-448b-a06f-5f599a932323 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:17.747107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd444ea-061f-448b-a06f-5f599a932323 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:17.747389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd444ea-061f-448b-a06f-5f599a932323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:17.897993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4dd444ea-061f-448b-a06f-5f599a932323 None None] MFA Rules not processed for user `429e173890b84ee289abc9793794ddb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:17.900070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4dd444ea-061f-448b-a06f-5f599a932323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:17.946072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:18.079851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 657/1310] 10.222.0.43 () {58 vars in 1061 bytes} [Mon Jul 4 07:55:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:18.094251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243fd92-d023-46f7-98d2-9e2e3d1243bb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:18.097756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6243fd92-d023-46f7-98d2-9e2e3d1243bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:18.281165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243fd92-d023-46f7-98d2-9e2e3d1243bb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:18.372345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8cdda-4cda-4064-af5e-e47a9f017478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:18.373034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8cdda-4cda-4064-af5e-e47a9f017478 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:18.373319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8cdda-4cda-4064-af5e-e47a9f017478 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:18.373598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8cdda-4cda-4064-af5e-e47a9f017478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:18.375465 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b0b8cdda-4cda-4064-af5e-e47a9f017478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:18.439876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:18.512500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8cdda-4cda-4064-af5e-e47a9f017478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:18.514414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8cdda-4cda-4064-af5e-e47a9f017478 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:18.540603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 654/1311] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:55:21.076890 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5179ac-f2c8-43fa-abd0-ae872737fb17 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:21.077804 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5179ac-f2c8-43fa-abd0-ae872737fb17 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:21.077804 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5179ac-f2c8-43fa-abd0-ae872737fb17 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:21.079363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 658/1312] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:21.086224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ae294-d2e7-447d-9721-9bd39276e1ef None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:21.086838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ae294-d2e7-447d-9721-9bd39276e1ef None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:21.087421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ae294-d2e7-447d-9721-9bd39276e1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:21.193059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e75ae294-d2e7-447d-9721-9bd39276e1ef None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:21.194497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e75ae294-d2e7-447d-9721-9bd39276e1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:21.305707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 655/1313] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 07:55:21.358470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0b9e3-42c5-43f6-9980-5c2e927a0d7f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:21.358997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0b9e3-42c5-43f6-9980-5c2e927a0d7f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:21.359410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0b9e3-42c5-43f6-9980-5c2e927a0d7f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:21.361342 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 659/1314] 10.222.0.43 () {54 vars in 964 bytes} [Mon Jul 4 07:55:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:21.369779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb980e49-764c-477f-9150-9abfe764514a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:21.370397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb980e49-764c-477f-9150-9abfe764514a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:21.370898 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb980e49-764c-477f-9150-9abfe764514a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:21.391529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:21.499722 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cb980e49-764c-477f-9150-9abfe764514a None None] MFA Rules not processed for user `429e173890b84ee289abc9793794ddb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:21.500981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cb980e49-764c-477f-9150-9abfe764514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:21.583782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 656/1315] 10.222.0.43 () {58 vars in 1061 bytes} [Mon Jul 4 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:21.593585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0b9e3-42c5-43f6-9980-5c2e927a0d7f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:21.595141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-57c0b9e3-42c5-43f6-9980-5c2e927a0d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:21.729446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0b9e3-42c5-43f6-9980-5c2e927a0d7f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:21.817649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:21.821826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5210c-3d46-4018-a4e9-b7cda835b694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:21.822806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5210c-3d46-4018-a4e9-b7cda835b694 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:21.823201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5210c-3d46-4018-a4e9-b7cda835b694 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:21.823981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5210c-3d46-4018-a4e9-b7cda835b694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:21.826232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d7f5210c-3d46-4018-a4e9-b7cda835b694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:21.961393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5210c-3d46-4018-a4e9-b7cda835b694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:21.962903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5210c-3d46-4018-a4e9-b7cda835b694 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:21.984613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 660/1316] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:55:24.626030 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63feb41-8b94-4fa7-bc8b-f9ca6ff130c3 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:24.626674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63feb41-8b94-4fa7-bc8b-f9ca6ff130c3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:24.626674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63feb41-8b94-4fa7-bc8b-f9ca6ff130c3 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:24.628066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 657/1317] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:24.634582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f7aa0-8f39-4332-abdd-fb4dc1321af6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:24.635138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f7aa0-8f39-4332-abdd-fb4dc1321af6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:24.635284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f7aa0-8f39-4332-abdd-fb4dc1321af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:24.714498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-647f7aa0-8f39-4332-abdd-fb4dc1321af6 None None] MFA Rules not processed for user `6c3cd73e4b2b4c3e83764570cbced9fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:24.715770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-647f7aa0-8f39-4332-abdd-fb4dc1321af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:24.778502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:24.861482 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 661/1318] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:24.902934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63feb41-8b94-4fa7-bc8b-f9ca6ff130c3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:24.904556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d63feb41-8b94-4fa7-bc8b-f9ca6ff130c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:25.070062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63feb41-8b94-4fa7-bc8b-f9ca6ff130c3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:25.075568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:25.167416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66684715-a59d-43af-a85a-adf41ab53cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:25.167999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66684715-a59d-43af-a85a-adf41ab53cfe service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:25.168291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66684715-a59d-43af-a85a-adf41ab53cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:25.168535 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66684715-a59d-43af-a85a-adf41ab53cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:25.170281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-66684715-a59d-43af-a85a-adf41ab53cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:25.293134 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66684715-a59d-43af-a85a-adf41ab53cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:25.294606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66684715-a59d-43af-a85a-adf41ab53cfe service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:25.314065 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 658/1319] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:24 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:55:28.020222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a219d63-552d-4ec5-be9e-fd67072c8417 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:28.020785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a219d63-552d-4ec5-be9e-fd67072c8417 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:28.020785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a219d63-552d-4ec5-be9e-fd67072c8417 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:28.022116 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 662/1320] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:28.027910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf55f7d-2ce9-489f-9aef-87f8a74f9b5b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:28.028395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf55f7d-2ce9-489f-9aef-87f8a74f9b5b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:28.028495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf55f7d-2ce9-489f-9aef-87f8a74f9b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:28.071935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:28.101593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-abf55f7d-2ce9-489f-9aef-87f8a74f9b5b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:28.102902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-abf55f7d-2ce9-489f-9aef-87f8a74f9b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:28.209282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 659/1321] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:28.249802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a219d63-552d-4ec5-be9e-fd67072c8417 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:28.251375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7a219d63-552d-4ec5-be9e-fd67072c8417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:28.365584 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a219d63-552d-4ec5-be9e-fd67072c8417 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:28.453564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a881e12-d1b2-4083-9009-1abcfbef659a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:28.454256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a881e12-d1b2-4083-9009-1abcfbef659a service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:28.454547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a881e12-d1b2-4083-9009-1abcfbef659a service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:28.454823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a881e12-d1b2-4083-9009-1abcfbef659a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:28.456843 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8a881e12-d1b2-4083-9009-1abcfbef659a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:28.465161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:28.576971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a881e12-d1b2-4083-9009-1abcfbef659a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:28.580032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a881e12-d1b2-4083-9009-1abcfbef659a service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:28.602647 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 663/1322] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 07:55:31.701921 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a1005-976a-482c-a9ad-48c426c40e84 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:31.702572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a1005-976a-482c-a9ad-48c426c40e84 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:31.702572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a1005-976a-482c-a9ad-48c426c40e84 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:31.704886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 660/1323] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:31.713203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbe5b5-e1b3-45f7-a147-19a7443d5932 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:31.713851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbe5b5-e1b3-45f7-a147-19a7443d5932 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:31.713978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbe5b5-e1b3-45f7-a147-19a7443d5932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:31.826210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-97bbe5b5-e1b3-45f7-a147-19a7443d5932 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:31.828207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-97bbe5b5-e1b3-45f7-a147-19a7443d5932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:31.913091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:31.961526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 664/1324] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:32.000286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a1005-976a-482c-a9ad-48c426c40e84 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:32.002694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-641a1005-976a-482c-a9ad-48c426c40e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:32.178069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a1005-976a-482c-a9ad-48c426c40e84 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:32.199274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:32.294043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae1e16-221c-4e86-88e0-ca993bc4102f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:32.294757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae1e16-221c-4e86-88e0-ca993bc4102f service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:32.295016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae1e16-221c-4e86-88e0-ca993bc4102f service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:32.295285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae1e16-221c-4e86-88e0-ca993bc4102f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:32.297088 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bbae1e16-221c-4e86-88e0-ca993bc4102f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:32.449795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae1e16-221c-4e86-88e0-ca993bc4102f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:32.451315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae1e16-221c-4e86-88e0-ca993bc4102f service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:32.476508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 661/1325] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:55:34.658863 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c030e2c-d6a5-4a8d-bad3-25cf91de09bb None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:34.659611 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c030e2c-d6a5-4a8d-bad3-25cf91de09bb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:34.659611 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c030e2c-d6a5-4a8d-bad3-25cf91de09bb None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:34.661094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 665/1326] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:34.667539 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ab202-a596-47e1-8709-6616fe5f0fc7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:34.668148 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ab202-a596-47e1-8709-6616fe5f0fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:34.668148 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ab202-a596-47e1-8709-6616fe5f0fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:34.726023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:34.748860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-445ab202-a596-47e1-8709-6616fe5f0fc7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:34.750179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-445ab202-a596-47e1-8709-6616fe5f0fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:34.861177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 662/1327] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:34.897588 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c030e2c-d6a5-4a8d-bad3-25cf91de09bb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:34.899172 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1c030e2c-d6a5-4a8d-bad3-25cf91de09bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:35.010769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c030e2c-d6a5-4a8d-bad3-25cf91de09bb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:35.091450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774a142-b5c1-42cb-a8fb-9027f7e613fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:35.092105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774a142-b5c1-42cb-a8fb-9027f7e613fd service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:35.092390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774a142-b5c1-42cb-a8fb-9027f7e613fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:35.092663 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774a142-b5c1-42cb-a8fb-9027f7e613fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:35.094391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d774a142-b5c1-42cb-a8fb-9027f7e613fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:35.113260 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:35.220276 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774a142-b5c1-42cb-a8fb-9027f7e613fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:35.221852 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774a142-b5c1-42cb-a8fb-9027f7e613fd service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:35.244349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 666/1328] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:55:37.374475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c48d9c-a0d3-40cc-b166-a3405e002131 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:37.375068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c48d9c-a0d3-40cc-b166-a3405e002131 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:37.375068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c48d9c-a0d3-40cc-b166-a3405e002131 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:37.376364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 663/1329] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:37.383226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbee1ec-8649-46fd-8f5c-7e1599f9a0db None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:37.383760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbee1ec-8649-46fd-8f5c-7e1599f9a0db None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:37.383760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbee1ec-8649-46fd-8f5c-7e1599f9a0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:37.458291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3dbee1ec-8649-46fd-8f5c-7e1599f9a0db None None] MFA Rules not processed for user `6c3cd73e4b2b4c3e83764570cbced9fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:37.459537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3dbee1ec-8649-46fd-8f5c-7e1599f9a0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:37.532895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:37.572537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 667/1330] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:37.610305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c48d9c-a0d3-40cc-b166-a3405e002131 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:37.611921 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-41c48d9c-a0d3-40cc-b166-a3405e002131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:37.730318 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c48d9c-a0d3-40cc-b166-a3405e002131 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:37.753226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:37.819629 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a8aed-0677-4488-97bb-20321a876806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:37.820295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a8aed-0677-4488-97bb-20321a876806 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:37.820578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a8aed-0677-4488-97bb-20321a876806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:37.820854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a8aed-0677-4488-97bb-20321a876806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:37.822573 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f39a8aed-0677-4488-97bb-20321a876806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:37.944720 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a8aed-0677-4488-97bb-20321a876806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:37.946345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a8aed-0677-4488-97bb-20321a876806 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:37.974153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 664/1331] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:55:40.552330 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faf039-d57d-462f-bfa4-eed6396d9707 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:40.552882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faf039-d57d-462f-bfa4-eed6396d9707 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:40.552882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faf039-d57d-462f-bfa4-eed6396d9707 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:40.554540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 668/1332] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:40.560419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae20b01-8ddc-42d0-8d35-f782bbf9a822 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:40.560991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae20b01-8ddc-42d0-8d35-f782bbf9a822 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:40.560991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae20b01-8ddc-42d0-8d35-f782bbf9a822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:40.631960 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:40.635068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4ae20b01-8ddc-42d0-8d35-f782bbf9a822 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:40.637147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4ae20b01-8ddc-42d0-8d35-f782bbf9a822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:40.744837 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 665/1333] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 07:55:40.784994 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faf039-d57d-462f-bfa4-eed6396d9707 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:40.787263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-48faf039-d57d-462f-bfa4-eed6396d9707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:40.902219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faf039-d57d-462f-bfa4-eed6396d9707 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:40.989214 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452729da-06d8-43cb-bbc0-cb09349cf725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:40.989913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452729da-06d8-43cb-bbc0-cb09349cf725 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:40.990199 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452729da-06d8-43cb-bbc0-cb09349cf725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:40.990475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452729da-06d8-43cb-bbc0-cb09349cf725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:40.992217 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-452729da-06d8-43cb-bbc0-cb09349cf725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:41.022767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:41.124728 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452729da-06d8-43cb-bbc0-cb09349cf725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:41.126304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452729da-06d8-43cb-bbc0-cb09349cf725 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:41.150537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 669/1334] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:55:43.333032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96334e3-46af-4929-b428-34ebf7c0bcd9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:43.333616 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96334e3-46af-4929-b428-34ebf7c0bcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:43.333616 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96334e3-46af-4929-b428-34ebf7c0bcd9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:43.335013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 666/1335] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:43.341208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e18fb-dc48-4d0b-8e9d-51289654d86c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:43.341838 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e18fb-dc48-4d0b-8e9d-51289654d86c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:43.341838 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e18fb-dc48-4d0b-8e9d-51289654d86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:43.426652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3d4e18fb-dc48-4d0b-8e9d-51289654d86c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:43.428922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3d4e18fb-dc48-4d0b-8e9d-51289654d86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:43.506329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:43.533016 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 670/1336] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:43.575321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96334e3-46af-4929-b428-34ebf7c0bcd9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:43.576899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e96334e3-46af-4929-b428-34ebf7c0bcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:43.687931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96334e3-46af-4929-b428-34ebf7c0bcd9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:43.719538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:43.774269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ba05a-e8c3-46ad-8e81-47dcddde5cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:43.774919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ba05a-e8c3-46ad-8e81-47dcddde5cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:43.775202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ba05a-e8c3-46ad-8e81-47dcddde5cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:43.775473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ba05a-e8c3-46ad-8e81-47dcddde5cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:43.777132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-144ba05a-e8c3-46ad-8e81-47dcddde5cf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:43.907780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ba05a-e8c3-46ad-8e81-47dcddde5cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:43.909343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ba05a-e8c3-46ad-8e81-47dcddde5cf3 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:43.929442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 667/1337] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:55:46.394750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f605a90-58cf-4898-a6ec-bc1ea222f0e9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:46.395353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f605a90-58cf-4898-a6ec-bc1ea222f0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:46.395353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f605a90-58cf-4898-a6ec-bc1ea222f0e9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:46.396636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 671/1338] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:46.403775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11fc9b-c15e-47e2-88b6-aba18e8489e8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:46.404287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11fc9b-c15e-47e2-88b6-aba18e8489e8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:46.404816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11fc9b-c15e-47e2-88b6-aba18e8489e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:46.504802 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5c11fc9b-c15e-47e2-88b6-aba18e8489e8 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:46.506463 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5c11fc9b-c15e-47e2-88b6-aba18e8489e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:46.515634 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:46.618620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 668/1339] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:46.657239 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f605a90-58cf-4898-a6ec-bc1ea222f0e9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:46.659007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7f605a90-58cf-4898-a6ec-bc1ea222f0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:46.776664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f605a90-58cf-4898-a6ec-bc1ea222f0e9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:46.855599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbd9bc-5fad-42f5-bf7b-e129c81a7739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:46.856280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbd9bc-5fad-42f5-bf7b-e129c81a7739 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:46.856565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbd9bc-5fad-42f5-bf7b-e129c81a7739 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:46.856838 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbd9bc-5fad-42f5-bf7b-e129c81a7739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:46.858621 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ecfbd9bc-5fad-42f5-bf7b-e129c81a7739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:46.899942 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:46.993264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbd9bc-5fad-42f5-bf7b-e129c81a7739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:46.994814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbd9bc-5fad-42f5-bf7b-e129c81a7739 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:47.017145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 672/1340] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:55:51.381609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccadea-c209-4005-95d0-4780f82c057d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:51.382182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccadea-c209-4005-95d0-4780f82c057d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:51.382182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccadea-c209-4005-95d0-4780f82c057d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:51.383609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 669/1341] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:51.389571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02521b1-2bb3-4dcd-a8e1-d7b0438e27be None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:51.390365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02521b1-2bb3-4dcd-a8e1-d7b0438e27be None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:51.390365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02521b1-2bb3-4dcd-a8e1-d7b0438e27be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:51.494830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f02521b1-2bb3-4dcd-a8e1-d7b0438e27be None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:51.496242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f02521b1-2bb3-4dcd-a8e1-d7b0438e27be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:51.637875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:51.665825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 673/1342] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:51.733116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccadea-c209-4005-95d0-4780f82c057d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:51.735909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-52ccadea-c209-4005-95d0-4780f82c057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:51.850374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccadea-c209-4005-95d0-4780f82c057d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:51.854485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3f714-b46c-4d37-bc9a-67f323cae98a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:51.855347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3f714-b46c-4d37-bc9a-67f323cae98a service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:51.855750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3f714-b46c-4d37-bc9a-67f323cae98a service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:51.856039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3f714-b46c-4d37-bc9a-67f323cae98a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:51.866920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3f714-b46c-4d37-bc9a-67f323cae98a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:51.868427 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3f714-b46c-4d37-bc9a-67f323cae98a service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:51.879088 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 670/1343] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:55:54.707558 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd63b7e-7774-4208-aabe-9114ac632414 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:54.708252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd63b7e-7774-4208-aabe-9114ac632414 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:54.708252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd63b7e-7774-4208-aabe-9114ac632414 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:54.709548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 674/1344] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:55:54.715598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c4e7c-cc51-420d-8539-fe2cd98316d6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:54.716139 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c4e7c-cc51-420d-8539-fe2cd98316d6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:54.716139 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c4e7c-cc51-420d-8539-fe2cd98316d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:54.747169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-166c4e7c-cc51-420d-8539-fe2cd98316d6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:54.748379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-166c4e7c-cc51-420d-8539-fe2cd98316d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:54.781283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:55:54.839052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 671/1345] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:54.878257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd63b7e-7774-4208-aabe-9114ac632414 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:54.880256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ebd63b7e-7774-4208-aabe-9114ac632414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:54.895735 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd63b7e-7774-4208-aabe-9114ac632414 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:54.899999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f55e2d-fe15-4dd6-962c-c930b3e977c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:54.900694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f55e2d-fe15-4dd6-962c-c930b3e977c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:54.900987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f55e2d-fe15-4dd6-962c-c930b3e977c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:54.901255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f55e2d-fe15-4dd6-962c-c930b3e977c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:54.913137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f55e2d-fe15-4dd6-962c-c930b3e977c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:54.914734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f55e2d-fe15-4dd6-962c-c930b3e977c5 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:54.927359 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 675/1346] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:55:59.799300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4a1f1-24f5-4c65-a77b-2c80b26a2cff None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:59.799763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4a1f1-24f5-4c65-a77b-2c80b26a2cff None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:59.799875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4a1f1-24f5-4c65-a77b-2c80b26a2cff None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:59.802604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 672/1347] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:55:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:55:59.808868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f356e86-ea4e-4626-a68c-040b10e1ae7c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:59.809394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f356e86-ea4e-4626-a68c-040b10e1ae7c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:59.809490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f356e86-ea4e-4626-a68c-040b10e1ae7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:59.828923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5f356e86-ea4e-4626-a68c-040b10e1ae7c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:55:59.831632 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5f356e86-ea4e-4626-a68c-040b10e1ae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:55:59.841796 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 676/1348] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:55:59.882098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4a1f1-24f5-4c65-a77b-2c80b26a2cff None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:55:59.894138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4a1f1-24f5-4c65-a77b-2c80b26a2cff None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:55:59.898486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b7230-2daf-458e-a0f6-4ba7ec21368f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:55:59.899144 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b7230-2daf-458e-a0f6-4ba7ec21368f service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:55:59.899424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b7230-2daf-458e-a0f6-4ba7ec21368f service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:55:59.899700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b7230-2daf-458e-a0f6-4ba7ec21368f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:55:59.909897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b7230-2daf-458e-a0f6-4ba7ec21368f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:55:59.911383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b7230-2daf-458e-a0f6-4ba7ec21368f service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:55:59.922216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 673/1349] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:05.070481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdbfff-02a7-4f34-b31f-a8ae276b2b4c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:05.071404 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdbfff-02a7-4f34-b31f-a8ae276b2b4c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:05.071831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdbfff-02a7-4f34-b31f-a8ae276b2b4c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:05.073463 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 677/1350] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 07:56:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:05.080186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22636d8b-d466-4280-b1c8-1638143098bc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:05.081910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22636d8b-d466-4280-b1c8-1638143098bc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:05.083580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22636d8b-d466-4280-b1c8-1638143098bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:05.200108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-22636d8b-d466-4280-b1c8-1638143098bc None None] MFA Rules not processed for user `f57dccfe9f5f4bc397f4c470fbf047dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:05.201705 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-22636d8b-d466-4280-b1c8-1638143098bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:05.354391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 674/1351] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 07:56:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:09.536508 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e69f0-a5e3-4857-b30d-007396efbd5e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:09.537063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e69f0-a5e3-4857-b30d-007396efbd5e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:09.537372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e69f0-a5e3-4857-b30d-007396efbd5e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:09.539548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 678/1352] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 07:56:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:09.545034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d4e89-fd2e-475b-b8c6-24d4eae929bc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:09.545490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d4e89-fd2e-475b-b8c6-24d4eae929bc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:09.545940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d4e89-fd2e-475b-b8c6-24d4eae929bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:09.565753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-636d4e89-fd2e-475b-b8c6-24d4eae929bc None None] MFA Rules not processed for user `f57dccfe9f5f4bc397f4c470fbf047dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:09.569606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-636d4e89-fd2e-475b-b8c6-24d4eae929bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:09.578794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 675/1353] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 07:56:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:09.823082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=13 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:56:11.344284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fe2c3-46c9-4b6e-9776-f0196b742138 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:11.344927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fe2c3-46c9-4b6e-9776-f0196b742138 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:11.344927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fe2c3-46c9-4b6e-9776-f0196b742138 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:11.346475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 679/1354] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 07:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:11.351806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54371496-c8e9-4234-9da5-28f02efeb78f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:11.352293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54371496-c8e9-4234-9da5-28f02efeb78f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:11.352389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54371496-c8e9-4234-9da5-28f02efeb78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:11.372560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-54371496-c8e9-4234-9da5-28f02efeb78f None None] MFA Rules not processed for user `f57dccfe9f5f4bc397f4c470fbf047dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:11.373706 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-54371496-c8e9-4234-9da5-28f02efeb78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:11.384126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 676/1355] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 07:56:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:14.936170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8741c0-0fae-4d22-9741-7cf805b6ee2b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:14.937304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8741c0-0fae-4d22-9741-7cf805b6ee2b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:14.937833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8741c0-0fae-4d22-9741-7cf805b6ee2b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:14.941295 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 680/1356] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:14.947424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c496d-b428-4e59-b3ac-e62331359720 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:14.947936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c496d-b428-4e59-b3ac-e62331359720 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:14.948047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c496d-b428-4e59-b3ac-e62331359720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:14.971891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-de4c496d-b428-4e59-b3ac-e62331359720 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:14.973588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-de4c496d-b428-4e59-b3ac-e62331359720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:14.983976 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 677/1357] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:15.070697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51889147-a0eb-41fe-a801-732c5584993a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:15.071267 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51889147-a0eb-41fe-a801-732c5584993a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:15.071672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51889147-a0eb-41fe-a801-732c5584993a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:15.073754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 681/1358] 10.222.0.43 () {54 vars in 961 bytes} [Mon Jul 4 07:56:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:15.083865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a5aad-367a-4c5c-b1b3-0343c29de220 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:15.084776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a5aad-367a-4c5c-b1b3-0343c29de220 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:15.085423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a5aad-367a-4c5c-b1b3-0343c29de220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:15.147328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:56:15.157261 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fc4a5aad-367a-4c5c-b1b3-0343c29de220 None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:15.158610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fc4a5aad-367a-4c5c-b1b3-0343c29de220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:15.250989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 678/1359] 10.222.0.43 () {58 vars in 1058 bytes} [Mon Jul 4 07:56:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:15.259605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51889147-a0eb-41fe-a801-732c5584993a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:15.277800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51889147-a0eb-41fe-a801-732c5584993a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:15.284079 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d967eeb-5a08-4528-a802-90b7b4788616 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:15.285162 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d967eeb-5a08-4528-a802-90b7b4788616 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:15.285561 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d967eeb-5a08-4528-a802-90b7b4788616 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:15.286000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d967eeb-5a08-4528-a802-90b7b4788616 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:15.295887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d967eeb-5a08-4528-a802-90b7b4788616 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:15.297787 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d967eeb-5a08-4528-a802-90b7b4788616 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:15.309398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 682/1360] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:17.586609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb57402-b87c-46e4-9720-ae7057ab419e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:17.587190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb57402-b87c-46e4-9720-ae7057ab419e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:17.587190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb57402-b87c-46e4-9720-ae7057ab419e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:17.588792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 679/1361] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:17.594547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a74019-fbdd-46d8-b0b3-a41d2a8ef7a1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:17.595064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a74019-fbdd-46d8-b0b3-a41d2a8ef7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:17.595526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a74019-fbdd-46d8-b0b3-a41d2a8ef7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:17.615610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-11a74019-fbdd-46d8-b0b3-a41d2a8ef7a1 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:17.616878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-11a74019-fbdd-46d8-b0b3-a41d2a8ef7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:17.625223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 683/1362] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:17.681460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb57402-b87c-46e4-9720-ae7057ab419e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:17.698566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb57402-b87c-46e4-9720-ae7057ab419e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:17.704409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43089b-16bb-4839-937c-004cea496f9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:17.705093 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43089b-16bb-4839-937c-004cea496f9d service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:17.705404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43089b-16bb-4839-937c-004cea496f9d service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:17.706009 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43089b-16bb-4839-937c-004cea496f9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:17.715729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43089b-16bb-4839-937c-004cea496f9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:17.717849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43089b-16bb-4839-937c-004cea496f9d service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:17.734125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 680/1363] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:20.136714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a993aa-0286-4a51-864d-a68683d5c1c8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:20.137220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a993aa-0286-4a51-864d-a68683d5c1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:20.137322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a993aa-0286-4a51-864d-a68683d5c1c8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:20.138834 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 684/1364] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:20.145728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3187ae-3739-4644-977d-e5ab8d33a6bb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:20.146300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3187ae-3739-4644-977d-e5ab8d33a6bb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:20.146300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3187ae-3739-4644-977d-e5ab8d33a6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:20.169043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3c3187ae-3739-4644-977d-e5ab8d33a6bb None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:20.170688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3c3187ae-3739-4644-977d-e5ab8d33a6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:20.183557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 681/1365] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:20.232878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b486c-bc50-4dda-9d4e-ca4f2fe1322f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:20.233234 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b486c-bc50-4dda-9d4e-ca4f2fe1322f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:20.233515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b486c-bc50-4dda-9d4e-ca4f2fe1322f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:20.234962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 685/1366] 10.222.0.43 () {54 vars in 961 bytes} [Mon Jul 4 07:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:20.241444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cfe6a-7bae-4d4a-866d-28794bdbe60f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:20.241813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cfe6a-7bae-4d4a-866d-28794bdbe60f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:20.242094 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cfe6a-7bae-4d4a-866d-28794bdbe60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:20.268027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-746cfe6a-7bae-4d4a-866d-28794bdbe60f None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:20.269571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-746cfe6a-7bae-4d4a-866d-28794bdbe60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:20.279985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 682/1367] 10.222.0.43 () {58 vars in 1058 bytes} [Mon Jul 4 07:56:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:20.288825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b486c-bc50-4dda-9d4e-ca4f2fe1322f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:20.318528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b486c-bc50-4dda-9d4e-ca4f2fe1322f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:20.326744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34c52f-4f87-43b4-b315-4f6808530844 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:20.327459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34c52f-4f87-43b4-b315-4f6808530844 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:20.327730 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34c52f-4f87-43b4-b315-4f6808530844 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:20.328008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34c52f-4f87-43b4-b315-4f6808530844 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:20.338334 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34c52f-4f87-43b4-b315-4f6808530844 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:20.340014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34c52f-4f87-43b4-b315-4f6808530844 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:20.351425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 686/1368] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 07:56:22.548663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610b70c-c73e-49cf-8f81-0d847632c41f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:22.549246 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610b70c-c73e-49cf-8f81-0d847632c41f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:22.549246 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610b70c-c73e-49cf-8f81-0d847632c41f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:22.551029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 683/1369] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:22.556568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6f5e7-57b3-479f-b079-3dc7ebf9bac6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:22.557040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6f5e7-57b3-479f-b079-3dc7ebf9bac6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:22.557121 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6f5e7-57b3-479f-b079-3dc7ebf9bac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:22.576091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8db6f5e7-57b3-479f-b079-3dc7ebf9bac6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:22.577317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8db6f5e7-57b3-479f-b079-3dc7ebf9bac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:22.586102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 687/1370] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:22.630914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610b70c-c73e-49cf-8f81-0d847632c41f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:22.646968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610b70c-c73e-49cf-8f81-0d847632c41f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:22.652880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863993a5-52cd-48c1-b379-50f49cf31c35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:22.653537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863993a5-52cd-48c1-b379-50f49cf31c35 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:22.653867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863993a5-52cd-48c1-b379-50f49cf31c35 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:22.654144 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863993a5-52cd-48c1-b379-50f49cf31c35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:22.663875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863993a5-52cd-48c1-b379-50f49cf31c35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:22.665507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863993a5-52cd-48c1-b379-50f49cf31c35 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:22.676347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 684/1371] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:23.350754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba12204-8e45-4232-9433-086d57d0de57 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:23.351031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba12204-8e45-4232-9433-086d57d0de57 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:23.351271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba12204-8e45-4232-9433-086d57d0de57 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:23.352838 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 688/1372] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:23.358634 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c234395-743f-4f66-9962-cf35177ccaa9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:23.358798 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c234395-743f-4f66-9962-cf35177ccaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:23.359035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c234395-743f-4f66-9962-cf35177ccaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:23.378110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3c234395-743f-4f66-9962-cf35177ccaa9 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:23.379363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3c234395-743f-4f66-9962-cf35177ccaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:23.388872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 685/1373] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 07:56:23.733340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42430be7-ae6a-4778-8957-1246d2133ca5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:23.733340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42430be7-ae6a-4778-8957-1246d2133ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:23.733340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42430be7-ae6a-4778-8957-1246d2133ca5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:23.735896 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 689/1374] 10.222.0.43 () {54 vars in 963 bytes} [Mon Jul 4 07:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:23.740991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe380fc-0b82-4304-897e-e2f730a3e5ef None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:23.741877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe380fc-0b82-4304-897e-e2f730a3e5ef None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:23.741877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe380fc-0b82-4304-897e-e2f730a3e5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:23.798797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6fe380fc-0b82-4304-897e-e2f730a3e5ef None None] MFA Rules not processed for user `061ffcf242ff4155b92daa8c3888fd10`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:23.800120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6fe380fc-0b82-4304-897e-e2f730a3e5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:23.883264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42430be7-ae6a-4778-8957-1246d2133ca5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:23.883701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 686/1375] 10.222.0.43 () {58 vars in 1060 bytes} [Mon Jul 4 07:56:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:23.895593 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42430be7-ae6a-4778-8957-1246d2133ca5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:23.899255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1f232-85b6-40f1-b9f7-3ef59e84a37d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:23.899974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1f232-85b6-40f1-b9f7-3ef59e84a37d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:23.900215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1f232-85b6-40f1-b9f7-3ef59e84a37d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:23.900427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1f232-85b6-40f1-b9f7-3ef59e84a37d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:23.910755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1f232-85b6-40f1-b9f7-3ef59e84a37d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:23.912228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1f232-85b6-40f1-b9f7-3ef59e84a37d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:23.922708 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 690/1376] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:24.813280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3988ef8-6760-452f-b78e-7eb7e360d6cb None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:24.813848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3988ef8-6760-452f-b78e-7eb7e360d6cb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:24.814102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3988ef8-6760-452f-b78e-7eb7e360d6cb None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:24.815351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 687/1377] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:56:24.823173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6b9d7-10b3-4259-adbd-6cb43100594d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:24.826256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6b9d7-10b3-4259-adbd-6cb43100594d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:24.826256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6b9d7-10b3-4259-adbd-6cb43100594d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:24.851820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7af6b9d7-10b3-4259-adbd-6cb43100594d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:24.853333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7af6b9d7-10b3-4259-adbd-6cb43100594d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:24.865847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 691/1378] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:24.913953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3988ef8-6760-452f-b78e-7eb7e360d6cb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:24.939069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3988ef8-6760-452f-b78e-7eb7e360d6cb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:24.946611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792df2b0-4635-4104-ba0c-59d55d546c86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:24.947377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792df2b0-4635-4104-ba0c-59d55d546c86 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:24.947731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792df2b0-4635-4104-ba0c-59d55d546c86 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:24.948080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792df2b0-4635-4104-ba0c-59d55d546c86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:24.959788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792df2b0-4635-4104-ba0c-59d55d546c86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:24.961698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792df2b0-4635-4104-ba0c-59d55d546c86 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:24.966048 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:56:24.974173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 688/1379] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:27.149051 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92657ca6-e256-4d81-b27a-608b94c62eca None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:27.149727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92657ca6-e256-4d81-b27a-608b94c62eca None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:27.149727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92657ca6-e256-4d81-b27a-608b94c62eca None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:27.151026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 692/1380] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:56:27.156539 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eb712-e2c5-40c5-a34c-e536f7113d26 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:27.156995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eb712-e2c5-40c5-a34c-e536f7113d26 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:27.157102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eb712-e2c5-40c5-a34c-e536f7113d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:27.176408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9e8eb712-e2c5-40c5-a34c-e536f7113d26 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:27.177674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9e8eb712-e2c5-40c5-a34c-e536f7113d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:27.186308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 689/1381] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:27.231832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92657ca6-e256-4d81-b27a-608b94c62eca None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:27.247914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92657ca6-e256-4d81-b27a-608b94c62eca None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:27.254902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67743e-4749-4aa8-9752-7629fac590e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:27.255570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67743e-4749-4aa8-9752-7629fac590e0 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:27.255852 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67743e-4749-4aa8-9752-7629fac590e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:27.256553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67743e-4749-4aa8-9752-7629fac590e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:27.265977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67743e-4749-4aa8-9752-7629fac590e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:27.267703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67743e-4749-4aa8-9752-7629fac590e0 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:27.278389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 693/1382] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:29.395942 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1beae8-ab8e-42ea-96d9-b22dcab36c52 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:29.396469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1beae8-ab8e-42ea-96d9-b22dcab36c52 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:29.396588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1beae8-ab8e-42ea-96d9-b22dcab36c52 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:29.397953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 690/1383] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:29.403579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5afed-ddf8-4f99-8ce0-385c009c06bc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:29.404022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5afed-ddf8-4f99-8ce0-385c009c06bc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:29.404104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5afed-ddf8-4f99-8ce0-385c009c06bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:29.422582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-55a5afed-ddf8-4f99-8ce0-385c009c06bc None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:29.423810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-55a5afed-ddf8-4f99-8ce0-385c009c06bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:29.431843 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 694/1384] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:29.474218 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1beae8-ab8e-42ea-96d9-b22dcab36c52 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:29.490453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1beae8-ab8e-42ea-96d9-b22dcab36c52 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:29.496146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a7d71-6eef-48d6-b8f0-00595da04d9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:29.496798 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a7d71-6eef-48d6-b8f0-00595da04d9f service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:29.497078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a7d71-6eef-48d6-b8f0-00595da04d9f service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:29.497351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a7d71-6eef-48d6-b8f0-00595da04d9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:29.507383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a7d71-6eef-48d6-b8f0-00595da04d9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:29.509215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a7d71-6eef-48d6-b8f0-00595da04d9f service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:29.520240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 691/1385] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:31.572861 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d7338-4c4d-47e5-a0fd-abe1ddde71b6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:31.573446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d7338-4c4d-47e5-a0fd-abe1ddde71b6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:31.573446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d7338-4c4d-47e5-a0fd-abe1ddde71b6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:31.574831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 695/1386] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:31.580716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c71c7-5987-4726-9c15-4f66a6565767 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:31.581258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c71c7-5987-4726-9c15-4f66a6565767 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:31.581406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c71c7-5987-4726-9c15-4f66a6565767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:31.600013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5a4c71c7-5987-4726-9c15-4f66a6565767 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:31.601272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5a4c71c7-5987-4726-9c15-4f66a6565767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:31.609304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 692/1387] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 07:56:31.654987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d7338-4c4d-47e5-a0fd-abe1ddde71b6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:31.672293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d7338-4c4d-47e5-a0fd-abe1ddde71b6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:31.679271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ba6a3-6899-45fb-b210-7d0cf5d264c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:31.680104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ba6a3-6899-45fb-b210-7d0cf5d264c5 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:31.680450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ba6a3-6899-45fb-b210-7d0cf5d264c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:31.680780 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ba6a3-6899-45fb-b210-7d0cf5d264c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:31.692043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ba6a3-6899-45fb-b210-7d0cf5d264c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:31.694092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ba6a3-6899-45fb-b210-7d0cf5d264c5 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:31.706315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 696/1388] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:33.831506 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527664a-3081-4cd3-b653-2b9c45dda9fd None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:33.832163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527664a-3081-4cd3-b653-2b9c45dda9fd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:33.832163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527664a-3081-4cd3-b653-2b9c45dda9fd None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:33.833536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 693/1389] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:33.839528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583469c-c5ef-4d2a-a7f5-8a03a94b560b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:33.840428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583469c-c5ef-4d2a-a7f5-8a03a94b560b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:33.840749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583469c-c5ef-4d2a-a7f5-8a03a94b560b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:33.864243 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d583469c-c5ef-4d2a-a7f5-8a03a94b560b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:33.865495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d583469c-c5ef-4d2a-a7f5-8a03a94b560b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:33.875082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 697/1390] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:33.920938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527664a-3081-4cd3-b653-2b9c45dda9fd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:33.936203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527664a-3081-4cd3-b653-2b9c45dda9fd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:33.942504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6457d4-2b2b-4455-b543-01e0d38a6a03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:33.943173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6457d4-2b2b-4455-b543-01e0d38a6a03 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:33.943455 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6457d4-2b2b-4455-b543-01e0d38a6a03 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:33.943735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6457d4-2b2b-4455-b543-01e0d38a6a03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:33.952778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6457d4-2b2b-4455-b543-01e0d38a6a03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:33.954837 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6457d4-2b2b-4455-b543-01e0d38a6a03 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:33.965350 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 694/1391] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 07:56:36.073196 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e68144-1ab4-42cb-b183-bbf592f7abc4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:36.073795 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e68144-1ab4-42cb-b183-bbf592f7abc4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:36.073795 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e68144-1ab4-42cb-b183-bbf592f7abc4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:36.075092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 698/1392] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:36.080752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf0a43-6c40-4cca-a9bb-0cd80580ab64 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:36.081262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf0a43-6c40-4cca-a9bb-0cd80580ab64 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:36.081365 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf0a43-6c40-4cca-a9bb-0cd80580ab64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:36.102260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fbbf0a43-6c40-4cca-a9bb-0cd80580ab64 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:36.103541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fbbf0a43-6c40-4cca-a9bb-0cd80580ab64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:36.112178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 695/1393] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 07:56:36.158697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e68144-1ab4-42cb-b183-bbf592f7abc4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:36.176252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e68144-1ab4-42cb-b183-bbf592f7abc4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:36.186903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a6902-69d6-4d36-b359-0fc80f88797e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:36.187810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a6902-69d6-4d36-b359-0fc80f88797e service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:36.188142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a6902-69d6-4d36-b359-0fc80f88797e service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:36.189036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a6902-69d6-4d36-b359-0fc80f88797e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:36.201758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a6902-69d6-4d36-b359-0fc80f88797e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:36.204060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a6902-69d6-4d36-b359-0fc80f88797e service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:36.218417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 699/1394] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:38.346149 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b3aa1-83a2-4860-88f9-a150c2e8d5d2 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:38.346788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b3aa1-83a2-4860-88f9-a150c2e8d5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:38.346788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b3aa1-83a2-4860-88f9-a150c2e8d5d2 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:38.348163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 696/1395] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:38.354430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90909f43-c6ef-4880-bc59-adad57a48643 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:38.354885 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90909f43-c6ef-4880-bc59-adad57a48643 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:38.355069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90909f43-c6ef-4880-bc59-adad57a48643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:38.376158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-90909f43-c6ef-4880-bc59-adad57a48643 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:38.377710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-90909f43-c6ef-4880-bc59-adad57a48643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:38.386457 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 700/1396] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:38.433188 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b3aa1-83a2-4860-88f9-a150c2e8d5d2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:38.449192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b3aa1-83a2-4860-88f9-a150c2e8d5d2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:38.455721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bc73a-3eb2-42f3-b9f9-8f4425c7879b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:38.456386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bc73a-3eb2-42f3-b9f9-8f4425c7879b service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:38.456673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bc73a-3eb2-42f3-b9f9-8f4425c7879b service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:38.456949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bc73a-3eb2-42f3-b9f9-8f4425c7879b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:38.465904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bc73a-3eb2-42f3-b9f9-8f4425c7879b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:38.467507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bc73a-3eb2-42f3-b9f9-8f4425c7879b service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:38.478302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 697/1397] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:39.176232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e289d8-35ed-493b-ae6b-c72d48ffb299 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:39.176515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e289d8-35ed-493b-ae6b-c72d48ffb299 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:39.176797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e289d8-35ed-493b-ae6b-c72d48ffb299 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:39.178563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 701/1398] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:39.184264 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e2f1a-120b-4321-a20a-58c322215764 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:39.184537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e2f1a-120b-4321-a20a-58c322215764 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:39.184810 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e2f1a-120b-4321-a20a-58c322215764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:39.204854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1a2e2f1a-120b-4321-a20a-58c322215764 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:39.206127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1a2e2f1a-120b-4321-a20a-58c322215764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:39.214462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 698/1399] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:39.276227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf29f6b-dd29-4245-bf99-38b661e8a97c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:39.276494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf29f6b-dd29-4245-bf99-38b661e8a97c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:39.276775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf29f6b-dd29-4245-bf99-38b661e8a97c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:39.278170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 702/1400] 10.222.0.43 () {54 vars in 962 bytes} [Mon Jul 4 07:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:39.283865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128da897-8f26-444d-ab53-35bf9d623ed1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:39.284243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128da897-8f26-444d-ab53-35bf9d623ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:39.284528 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128da897-8f26-444d-ab53-35bf9d623ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:39.340588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-128da897-8f26-444d-ab53-35bf9d623ed1 None None] MFA Rules not processed for user `4a122422a45446c9a63d67b4f931fa47`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:39.341919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-128da897-8f26-444d-ab53-35bf9d623ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:39.408007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 699/1401] 10.222.0.43 () {58 vars in 1059 bytes} [Mon Jul 4 07:56:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:39.415579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf29f6b-dd29-4245-bf99-38b661e8a97c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:39.427414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf29f6b-dd29-4245-bf99-38b661e8a97c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:39.431119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e5023-f448-45ea-99d1-33a7c1d2f159 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a122422a45446c9a63d67b4f931fa47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:39.431786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e5023-f448-45ea-99d1-33a7c1d2f159 service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:39.432067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e5023-f448-45ea-99d1-33a7c1d2f159 service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:39.432340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e5023-f448-45ea-99d1-33a7c1d2f159 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:39.442439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e5023-f448-45ea-99d1-33a7c1d2f159 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:39.443921 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e5023-f448-45ea-99d1-33a7c1d2f159 service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:39.454394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 703/1402] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:40.200314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612355e-382e-4e5c-bb90-2775ed61b72b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:40.200743 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612355e-382e-4e5c-bb90-2775ed61b72b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:40.200860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612355e-382e-4e5c-bb90-2775ed61b72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:40.254367 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c612355e-382e-4e5c-bb90-2775ed61b72b None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:40.255575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c612355e-382e-4e5c-bb90-2775ed61b72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:40.299545 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:56:40.327357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 700/1403] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 07:56:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:40.371854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66365fe4-02bf-4cde-8c50-3683d0f92c08 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:40.372156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66365fe4-02bf-4cde-8c50-3683d0f92c08 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:40.372426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66365fe4-02bf-4cde-8c50-3683d0f92c08 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:40.373858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 704/1404] 10.222.0.43 () {54 vars in 942 bytes} [Mon Jul 4 07:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:56:40.381775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1579e5-68cb-4f38-8988-ebc47275117a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:40.382140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1579e5-68cb-4f38-8988-ebc47275117a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:40.382490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1579e5-68cb-4f38-8988-ebc47275117a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:40.434319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bb1579e5-68cb-4f38-8988-ebc47275117a None None] MFA Rules not processed for user `6763884b51204c488ca8c7e452e3a29e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:40.435586 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bb1579e5-68cb-4f38-8988-ebc47275117a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:40.515891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 701/1405] 10.222.0.43 () {58 vars in 1039 bytes} [Mon Jul 4 07:56:40 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:40.525980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66365fe4-02bf-4cde-8c50-3683d0f92c08 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:40.534041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=61 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:56:40.542008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66365fe4-02bf-4cde-8c50-3683d0f92c08 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:40.547495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54400c8-03de-499a-8a47-c17e718128e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:40.548207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54400c8-03de-499a-8a47-c17e718128e9 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:40.548662 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54400c8-03de-499a-8a47-c17e718128e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:40.548946 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54400c8-03de-499a-8a47-c17e718128e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:40.558013 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54400c8-03de-499a-8a47-c17e718128e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:40.559645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54400c8-03de-499a-8a47-c17e718128e9 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:40.568689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 705/1406] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 07:56:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 07:56:40.643941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbcb0-276c-480f-aa13-2f116c14a958 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:40.644354 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbcb0-276c-480f-aa13-2f116c14a958 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:40.644755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbcb0-276c-480f-aa13-2f116c14a958 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:40.647280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 702/1407] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:56:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:56:40.653318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18924e3-4451-46cd-b2a2-31b7b0c63160 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:40.653651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18924e3-4451-46cd-b2a2-31b7b0c63160 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:40.653970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18924e3-4451-46cd-b2a2-31b7b0c63160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:40.677182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c18924e3-4451-46cd-b2a2-31b7b0c63160 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:56:40.678588 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c18924e3-4451-46cd-b2a2-31b7b0c63160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:56:40.689968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 706/1408] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:56:40.739321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bbcb0-276c-480f-aa13-2f116c14a958 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:56:40.755488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bbcb0-276c-480f-aa13-2f116c14a958 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:56:40.762412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49e415-19cd-4fe8-98e9-644a3372cd2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:56:40.763100 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49e415-19cd-4fe8-98e9-644a3372cd2b service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:56:40.763384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49e415-19cd-4fe8-98e9-644a3372cd2b service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:56:40.763662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49e415-19cd-4fe8-98e9-644a3372cd2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:56:40.774762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49e415-19cd-4fe8-98e9-644a3372cd2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:56:40.776406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49e415-19cd-4fe8-98e9-644a3372cd2b service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:56:40.788020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 703/1409] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:56:50.668266 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=8 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:56:50.783580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=59 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:57:02.539290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d7ca6-b387-41b5-94d4-0714b2d4d9d7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:02.539907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d7ca6-b387-41b5-94d4-0714b2d4d9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:02.539907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d7ca6-b387-41b5-94d4-0714b2d4d9d7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:02.541240 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 707/1410] 10.222.0.43 () {54 vars in 918 bytes} [Mon Jul 4 07:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:02.546605 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5209e-9dae-47d2-bf56-a7f3a9bdf27d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:02.547044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5209e-9dae-47d2-bf56-a7f3a9bdf27d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:02.547172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5209e-9dae-47d2-bf56-a7f3a9bdf27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:02.567022 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1da5209e-9dae-47d2-bf56-a7f3a9bdf27d None None] MFA Rules not processed for user `f57dccfe9f5f4bc397f4c470fbf047dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:02.568258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1da5209e-9dae-47d2-bf56-a7f3a9bdf27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:02.578473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 704/1411] 10.222.0.43 () {58 vars in 1015 bytes} [Mon Jul 4 07:57:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:02.593451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578ef16-9c24-48bb-8a76-555409fc65b2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:02.593780 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578ef16-9c24-48bb-8a76-555409fc65b2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:02.594179 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578ef16-9c24-48bb-8a76-555409fc65b2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:02.595605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 708/1412] 10.222.0.43 () {54 vars in 918 bytes} [Mon Jul 4 07:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:02.600800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1901265-7323-4d4c-a508-0e14a3aa4ebe None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:02.601092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1901265-7323-4d4c-a508-0e14a3aa4ebe None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:02.601373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1901265-7323-4d4c-a508-0e14a3aa4ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:02.621035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a1901265-7323-4d4c-a508-0e14a3aa4ebe None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:02.622290 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a1901265-7323-4d4c-a508-0e14a3aa4ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:02.633353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 705/1413] 10.222.0.43 () {58 vars in 1015 bytes} [Mon Jul 4 07:57:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 07:57:05.179528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5a435-470b-486b-bc7b-7d89526e55ee None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:05.180105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5a435-470b-486b-bc7b-7d89526e55ee None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:05.180105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5a435-470b-486b-bc7b-7d89526e55ee None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:05.181986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 07:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:05.188674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d1a09-6d02-488c-a661-682572c9e560 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:05.189216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d1a09-6d02-488c-a661-682572c9e560 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:05.189216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d1a09-6d02-488c-a661-682572c9e560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:05.207927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-882d1a09-6d02-488c-a661-682572c9e560 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:05.209143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-882d1a09-6d02-488c-a661-682572c9e560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:05.217413 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 07:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 07:57:05.261099 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5a435-470b-486b-bc7b-7d89526e55ee None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:05.276564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5a435-470b-486b-bc7b-7d89526e55ee None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:05.281891 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46f54c-ea9a-40a6-9050-5062a75c6f2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:05.282533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46f54c-ea9a-40a6-9050-5062a75c6f2e service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:05.282810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46f54c-ea9a-40a6-9050-5062a75c6f2e service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:05.283085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46f54c-ea9a-40a6-9050-5062a75c6f2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:05.292023 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46f54c-ea9a-40a6-9050-5062a75c6f2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:05.293648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46f54c-ea9a-40a6-9050-5062a75c6f2e service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:05.304335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 710/1416] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:06.230405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ac306-9559-4335-860d-63f8faf27c1f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:06.231002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ac306-9559-4335-860d-63f8faf27c1f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:06.231002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ac306-9559-4335-860d-63f8faf27c1f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:06.232331 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 07:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:06.237823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ea41c-df80-41c4-86c2-dda92c29fe76 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:06.238303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ea41c-df80-41c4-86c2-dda92c29fe76 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:06.238397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ea41c-df80-41c4-86c2-dda92c29fe76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:06.258208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-de2ea41c-df80-41c4-86c2-dda92c29fe76 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:06.259442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-de2ea41c-df80-41c4-86c2-dda92c29fe76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:06.267653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 07:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:06.417765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ac306-9559-4335-860d-63f8faf27c1f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:06.432868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ac306-9559-4335-860d-63f8faf27c1f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:06.438654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab5296-b96d-44f5-ba6a-7b775b4895dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:06.439298 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab5296-b96d-44f5-ba6a-7b775b4895dc service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:06.439576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab5296-b96d-44f5-ba6a-7b775b4895dc service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:06.439872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab5296-b96d-44f5-ba6a-7b775b4895dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:06.449264 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab5296-b96d-44f5-ba6a-7b775b4895dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:06.450935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab5296-b96d-44f5-ba6a-7b775b4895dc service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:06.461889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 708/1419] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:07.342075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b0004-eb1e-4881-8c1d-4050eab01b18 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:07.342700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b0004-eb1e-4881-8c1d-4050eab01b18 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:07.342700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b0004-eb1e-4881-8c1d-4050eab01b18 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:07.344169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 07:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:07.349464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99c8fb-90bc-40ec-a024-85553a19f0b8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:07.349976 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99c8fb-90bc-40ec-a024-85553a19f0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:07.350077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99c8fb-90bc-40ec-a024-85553a19f0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:07.370540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-da99c8fb-90bc-40ec-a024-85553a19f0b8 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:07.371680 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-da99c8fb-90bc-40ec-a024-85553a19f0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:07.381316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 07:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:07.531944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b0004-eb1e-4881-8c1d-4050eab01b18 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:07.547551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b0004-eb1e-4881-8c1d-4050eab01b18 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:07.553966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0de30-7cc7-48b2-acee-de9724a6a113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:07.554618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0de30-7cc7-48b2-acee-de9724a6a113 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:07.554923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0de30-7cc7-48b2-acee-de9724a6a113 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:07.555179 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0de30-7cc7-48b2-acee-de9724a6a113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:07.564474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0de30-7cc7-48b2-acee-de9724a6a113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:07.566109 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0de30-7cc7-48b2-acee-de9724a6a113 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:07.576184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 713/1422] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:08.449289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600e4d5-0078-4a33-83b4-428174dddf28 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:08.449877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600e4d5-0078-4a33-83b4-428174dddf28 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:08.449877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600e4d5-0078-4a33-83b4-428174dddf28 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:08.451157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 07:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:08.456399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740aa06-be8b-467e-8315-51308007e20d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:08.456899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740aa06-be8b-467e-8315-51308007e20d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:08.456899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740aa06-be8b-467e-8315-51308007e20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:08.477026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3740aa06-be8b-467e-8315-51308007e20d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:08.478295 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3740aa06-be8b-467e-8315-51308007e20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:08.487680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 07:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:08.637644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600e4d5-0078-4a33-83b4-428174dddf28 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:08.653643 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600e4d5-0078-4a33-83b4-428174dddf28 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:08.659956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc486e-208c-4283-b4c8-f7fa05e55dc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:08.660589 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc486e-208c-4283-b4c8-f7fa05e55dc5 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:08.660867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc486e-208c-4283-b4c8-f7fa05e55dc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:08.661138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc486e-208c-4283-b4c8-f7fa05e55dc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:08.669644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc486e-208c-4283-b4c8-f7fa05e55dc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:08.671228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc486e-208c-4283-b4c8-f7fa05e55dc5 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:08.681006 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 711/1425] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:09.552028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c349d-d086-4190-b31a-9b5d185a9fe6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:09.552633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c349d-d086-4190-b31a-9b5d185a9fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:09.552633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c349d-d086-4190-b31a-9b5d185a9fe6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:09.553999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 07:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:09.559189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5570f6-68e6-4e5e-a637-5e427b017064 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:09.559704 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5570f6-68e6-4e5e-a637-5e427b017064 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:09.559704 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5570f6-68e6-4e5e-a637-5e427b017064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:09.579357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-dc5570f6-68e6-4e5e-a637-5e427b017064 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:09.580583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-dc5570f6-68e6-4e5e-a637-5e427b017064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:09.589368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 07:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:09.739212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c349d-d086-4190-b31a-9b5d185a9fe6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:09.756165 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c349d-d086-4190-b31a-9b5d185a9fe6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:09.762188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4b605-f00d-46d8-afb5-052424bcf373 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:09.762832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4b605-f00d-46d8-afb5-052424bcf373 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:09.763115 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4b605-f00d-46d8-afb5-052424bcf373 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:09.763397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4b605-f00d-46d8-afb5-052424bcf373 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:09.773049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4b605-f00d-46d8-afb5-052424bcf373 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:09.774671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4b605-f00d-46d8-afb5-052424bcf373 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:09.784577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 716/1428] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:10.602516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ede8e-3e68-4f40-97dc-ed98e3381681 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:10.603340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ede8e-3e68-4f40-97dc-ed98e3381681 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:10.603340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ede8e-3e68-4f40-97dc-ed98e3381681 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:10.604601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 07:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:10.610714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9feef8-be91-4f22-bd44-0cc9ed8f364f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:10.611277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9feef8-be91-4f22-bd44-0cc9ed8f364f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:10.611277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9feef8-be91-4f22-bd44-0cc9ed8f364f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:10.632629 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8e9feef8-be91-4f22-bd44-0cc9ed8f364f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:10.633928 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8e9feef8-be91-4f22-bd44-0cc9ed8f364f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:10.642715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 07:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:10.798683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ede8e-3e68-4f40-97dc-ed98e3381681 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:10.814568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ede8e-3e68-4f40-97dc-ed98e3381681 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:10.820472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad1703-22b3-4c80-b76b-68b01b36df55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:10.821156 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad1703-22b3-4c80-b76b-68b01b36df55 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:10.821428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad1703-22b3-4c80-b76b-68b01b36df55 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:10.821737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad1703-22b3-4c80-b76b-68b01b36df55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:10.831366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad1703-22b3-4c80-b76b-68b01b36df55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:10.833010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad1703-22b3-4c80-b76b-68b01b36df55 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:10.843658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 714/1431] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:11.681220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ac9ff-9686-4551-860f-e7d7122d777e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:11.681840 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ac9ff-9686-4551-860f-e7d7122d777e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:11.681840 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ac9ff-9686-4551-860f-e7d7122d777e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:11.683166 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 07:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:11.689876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cd685-f96a-4e69-8356-7a5cb9607113 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:11.690679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cd685-f96a-4e69-8356-7a5cb9607113 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:11.690848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cd685-f96a-4e69-8356-7a5cb9607113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:11.716382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c36cd685-f96a-4e69-8356-7a5cb9607113 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:11.718084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c36cd685-f96a-4e69-8356-7a5cb9607113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:11.726441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 07:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:11.879479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ac9ff-9686-4551-860f-e7d7122d777e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:11.896446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ac9ff-9686-4551-860f-e7d7122d777e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:11.902304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e948f2-4277-4617-8bbc-d6cce8bedf2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:11.902971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e948f2-4277-4617-8bbc-d6cce8bedf2d service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:11.903259 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e948f2-4277-4617-8bbc-d6cce8bedf2d service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:11.903535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e948f2-4277-4617-8bbc-d6cce8bedf2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:11.913985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e948f2-4277-4617-8bbc-d6cce8bedf2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:11.915644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e948f2-4277-4617-8bbc-d6cce8bedf2d service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:11.926831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 719/1434] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:12.791148 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddee57f-5296-4aef-9081-5c17819cd9e1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:12.791736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddee57f-5296-4aef-9081-5c17819cd9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:12.791736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddee57f-5296-4aef-9081-5c17819cd9e1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:12.793086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 07:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:12.799355 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3b669-d537-4100-8df7-171be68f1146 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:12.799905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3b669-d537-4100-8df7-171be68f1146 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:12.800007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3b669-d537-4100-8df7-171be68f1146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:12.821289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f3e3b669-d537-4100-8df7-171be68f1146 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:12.822561 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f3e3b669-d537-4100-8df7-171be68f1146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:12.832098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 07:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:12.992662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddee57f-5296-4aef-9081-5c17819cd9e1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:13.010035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddee57f-5296-4aef-9081-5c17819cd9e1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:13.018498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0face6-0068-44fb-9e5d-8df94ca03955 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:13.019353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0face6-0068-44fb-9e5d-8df94ca03955 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:13.019825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0face6-0068-44fb-9e5d-8df94ca03955 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:13.020221 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0face6-0068-44fb-9e5d-8df94ca03955 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:13.032318 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0face6-0068-44fb-9e5d-8df94ca03955 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:13.034043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0face6-0068-44fb-9e5d-8df94ca03955 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:13.048213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 717/1437] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:13.860978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc917-5cb4-4a37-a03a-4db3860c266c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:13.861560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc917-5cb4-4a37-a03a-4db3860c266c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:13.861560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc917-5cb4-4a37-a03a-4db3860c266c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:13.862969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 07:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:13.868373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bef91-8f88-4bac-a914-f8fb33ee0c41 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:13.868873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bef91-8f88-4bac-a914-f8fb33ee0c41 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:13.868970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bef91-8f88-4bac-a914-f8fb33ee0c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:13.889013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-df2bef91-8f88-4bac-a914-f8fb33ee0c41 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:13.890553 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-df2bef91-8f88-4bac-a914-f8fb33ee0c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:13.899592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 07:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:14.051220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edc917-5cb4-4a37-a03a-4db3860c266c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:14.075092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edc917-5cb4-4a37-a03a-4db3860c266c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:14.082026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036dcbb-b274-42e4-8954-cf65ab8c4680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:14.082834 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036dcbb-b274-42e4-8954-cf65ab8c4680 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:14.083114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036dcbb-b274-42e4-8954-cf65ab8c4680 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:14.083449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036dcbb-b274-42e4-8954-cf65ab8c4680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:14.095717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036dcbb-b274-42e4-8954-cf65ab8c4680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:14.097366 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036dcbb-b274-42e4-8954-cf65ab8c4680 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:14.108377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 722/1440] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:16.069504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeff1af-3906-4cc8-afb8-bfebd8821487 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:16.070151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeff1af-3906-4cc8-afb8-bfebd8821487 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:16.070151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeff1af-3906-4cc8-afb8-bfebd8821487 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:16.071385 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 07:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:57:16.077163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc14c-db81-4e32-afa3-194ff698c2c6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:16.078075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc14c-db81-4e32-afa3-194ff698c2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:16.078371 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc14c-db81-4e32-afa3-194ff698c2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:16.099067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e8cdc14c-db81-4e32-afa3-194ff698c2c6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:16.100312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e8cdc14c-db81-4e32-afa3-194ff698c2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:16.110558 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 07:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:16.150697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeff1af-3906-4cc8-afb8-bfebd8821487 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:16.162682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeff1af-3906-4cc8-afb8-bfebd8821487 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:16.167128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f3d09-d6fa-478f-bdfe-2cc8967a015c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:16.167780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f3d09-d6fa-478f-bdfe-2cc8967a015c service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:16.168060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f3d09-d6fa-478f-bdfe-2cc8967a015c service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:16.168332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f3d09-d6fa-478f-bdfe-2cc8967a015c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:16.179489 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f3d09-d6fa-478f-bdfe-2cc8967a015c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:16.180465 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f3d09-d6fa-478f-bdfe-2cc8967a015c service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:16.191520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 720/1443] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:18.425786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cce56-7b2b-407a-9952-f445386fb91a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:18.426608 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cce56-7b2b-407a-9952-f445386fb91a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:18.426608 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cce56-7b2b-407a-9952-f445386fb91a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:18.428677 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 07:57:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:18.437269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d400c12-ead8-4795-88b9-9c64fddf1368 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:18.437716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d400c12-ead8-4795-88b9-9c64fddf1368 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:18.437867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d400c12-ead8-4795-88b9-9c64fddf1368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:18.458219 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0d400c12-ead8-4795-88b9-9c64fddf1368 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:18.459443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0d400c12-ead8-4795-88b9-9c64fddf1368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:18.469257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 07:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:18.508242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cce56-7b2b-407a-9952-f445386fb91a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:18.521105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cce56-7b2b-407a-9952-f445386fb91a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:18.525394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7838ad-98b6-42e2-bfae-ecdb3f01d23c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:18.526056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7838ad-98b6-42e2-bfae-ecdb3f01d23c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:18.526332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7838ad-98b6-42e2-bfae-ecdb3f01d23c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:18.526632 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7838ad-98b6-42e2-bfae-ecdb3f01d23c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:18.536333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7838ad-98b6-42e2-bfae-ecdb3f01d23c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:18.537838 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7838ad-98b6-42e2-bfae-ecdb3f01d23c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:18.548086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 725/1446] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:21.020583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec87967-d3dc-43bc-a380-92a9faacf402 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:21.021508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec87967-d3dc-43bc-a380-92a9faacf402 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:21.022393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec87967-d3dc-43bc-a380-92a9faacf402 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:21.026910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 07:57:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:21.036236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb98237-7bd1-458b-901c-92ed4f0e6b3b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:21.037020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb98237-7bd1-458b-901c-92ed4f0e6b3b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:21.037248 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb98237-7bd1-458b-901c-92ed4f0e6b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:21.060091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2eb98237-7bd1-458b-901c-92ed4f0e6b3b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:21.061574 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2eb98237-7bd1-458b-901c-92ed4f0e6b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:21.070505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 07:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:21.110773 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec87967-d3dc-43bc-a380-92a9faacf402 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:21.127246 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec87967-d3dc-43bc-a380-92a9faacf402 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:21.134128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363ef24-19df-4399-ad5a-33e41438a5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:21.135126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363ef24-19df-4399-ad5a-33e41438a5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:21.135540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363ef24-19df-4399-ad5a-33e41438a5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:21.135900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363ef24-19df-4399-ad5a-33e41438a5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:21.148032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363ef24-19df-4399-ad5a-33e41438a5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:21.149477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363ef24-19df-4399-ad5a-33e41438a5d2 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:21.161806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 723/1449] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:23.383212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2184f7-4847-42bf-915e-13eccfaa5505 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:23.383770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2184f7-4847-42bf-915e-13eccfaa5505 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:23.383770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2184f7-4847-42bf-915e-13eccfaa5505 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:23.385090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 07:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:23.390983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4f72c-d468-40ce-8f51-f7d4d0c983bb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:23.391402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4f72c-d468-40ce-8f51-f7d4d0c983bb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:23.391541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4f72c-d468-40ce-8f51-f7d4d0c983bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:23.410792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-dbc4f72c-d468-40ce-8f51-f7d4d0c983bb None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:23.412011 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-dbc4f72c-d468-40ce-8f51-f7d4d0c983bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:23.420591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 07:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:23.464532 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2184f7-4847-42bf-915e-13eccfaa5505 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:23.476729 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2184f7-4847-42bf-915e-13eccfaa5505 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:23.480389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6262e9-1692-42da-9b9d-f139c991bf40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:23.481043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6262e9-1692-42da-9b9d-f139c991bf40 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:23.481322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6262e9-1692-42da-9b9d-f139c991bf40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:23.481608 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6262e9-1692-42da-9b9d-f139c991bf40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:23.492551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6262e9-1692-42da-9b9d-f139c991bf40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:23.494047 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6262e9-1692-42da-9b9d-f139c991bf40 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:23.504181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 728/1452] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 07:57:25.696060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27060cf-b9bd-4191-bbbb-d18104b25a6e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:25.696701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27060cf-b9bd-4191-bbbb-d18104b25a6e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:25.696701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27060cf-b9bd-4191-bbbb-d18104b25a6e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:25.698067 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 07:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:25.705402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d687ce-c86c-4e6e-a95d-d7bbe5ac62fd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:25.706192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d687ce-c86c-4e6e-a95d-d7bbe5ac62fd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:25.706387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d687ce-c86c-4e6e-a95d-d7bbe5ac62fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:25.730373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a8d687ce-c86c-4e6e-a95d-d7bbe5ac62fd None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:25.731605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a8d687ce-c86c-4e6e-a95d-d7bbe5ac62fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:25.741339 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 07:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:25.780453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27060cf-b9bd-4191-bbbb-d18104b25a6e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:25.792291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27060cf-b9bd-4191-bbbb-d18104b25a6e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:25.796320 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e2d5a-59fa-4aff-b5cc-64a258bd3154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:25.797021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e2d5a-59fa-4aff-b5cc-64a258bd3154 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:25.797266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e2d5a-59fa-4aff-b5cc-64a258bd3154 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:25.797517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e2d5a-59fa-4aff-b5cc-64a258bd3154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:25.808734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e2d5a-59fa-4aff-b5cc-64a258bd3154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:25.810249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e2d5a-59fa-4aff-b5cc-64a258bd3154 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:25.820999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 726/1455] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:27.913407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e450b-a7d3-4b3d-8e2e-01aac99c2be7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:27.914020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e450b-a7d3-4b3d-8e2e-01aac99c2be7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:27.914020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e450b-a7d3-4b3d-8e2e-01aac99c2be7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:27.915388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 07:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:27.921576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e442dc5-8a32-4ea8-8b59-8ba0ee63fab7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:27.922224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e442dc5-8a32-4ea8-8b59-8ba0ee63fab7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:27.922360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e442dc5-8a32-4ea8-8b59-8ba0ee63fab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:27.949927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6e442dc5-8a32-4ea8-8b59-8ba0ee63fab7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:27.951819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6e442dc5-8a32-4ea8-8b59-8ba0ee63fab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:27.966505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 07:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:28.017310 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e450b-a7d3-4b3d-8e2e-01aac99c2be7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:28.034015 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e450b-a7d3-4b3d-8e2e-01aac99c2be7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:28.038046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab02ff-9988-4831-8991-1e92d3cebda0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:28.038684 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab02ff-9988-4831-8991-1e92d3cebda0 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:28.038963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab02ff-9988-4831-8991-1e92d3cebda0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:28.039234 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab02ff-9988-4831-8991-1e92d3cebda0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:28.048708 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab02ff-9988-4831-8991-1e92d3cebda0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:28.050139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab02ff-9988-4831-8991-1e92d3cebda0 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:28.059708 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 731/1458] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:30.203415 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69e7b7-0384-4388-9fb4-ea1733273b9d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:30.204344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69e7b7-0384-4388-9fb4-ea1733273b9d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:30.204344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69e7b7-0384-4388-9fb4-ea1733273b9d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:30.206515 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 07:57:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:30.212804 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98957b9e-b780-4c9e-9932-ec7f9285f280 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:30.213253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98957b9e-b780-4c9e-9932-ec7f9285f280 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:30.213378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98957b9e-b780-4c9e-9932-ec7f9285f280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:30.241168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-98957b9e-b780-4c9e-9932-ec7f9285f280 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:30.243008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-98957b9e-b780-4c9e-9932-ec7f9285f280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:30.254220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 07:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:30.305554 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69e7b7-0384-4388-9fb4-ea1733273b9d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:30.320281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69e7b7-0384-4388-9fb4-ea1733273b9d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:30.324461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda41ee6-0f10-4844-b928-5c69d27cf797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:30.325129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda41ee6-0f10-4844-b928-5c69d27cf797 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:30.325393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda41ee6-0f10-4844-b928-5c69d27cf797 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:30.325662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda41ee6-0f10-4844-b928-5c69d27cf797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:30.336523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda41ee6-0f10-4844-b928-5c69d27cf797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:30.338065 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda41ee6-0f10-4844-b928-5c69d27cf797 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:30.348104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 729/1461] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:32.460313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab30778-5b18-4245-ae08-1cab4f7cea68 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:32.460915 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab30778-5b18-4245-ae08-1cab4f7cea68 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:32.460915 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab30778-5b18-4245-ae08-1cab4f7cea68 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:32.462211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 07:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:32.468146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8b56e-f74c-46db-b654-e0fcfe6534a0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:32.468738 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8b56e-f74c-46db-b654-e0fcfe6534a0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:32.468738 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8b56e-f74c-46db-b654-e0fcfe6534a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:32.476242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=79 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:57:32.491273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f9c8b56e-f74c-46db-b654-e0fcfe6534a0 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:32.492492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f9c8b56e-f74c-46db-b654-e0fcfe6534a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:32.503369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 07:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:32.554039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab30778-5b18-4245-ae08-1cab4f7cea68 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:32.562674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=73 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:57:32.569839 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab30778-5b18-4245-ae08-1cab4f7cea68 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:32.574145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46e2dc-0ee7-434a-a2c7-d941aab5d340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:32.574786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46e2dc-0ee7-434a-a2c7-d941aab5d340 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:32.575074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46e2dc-0ee7-434a-a2c7-d941aab5d340 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:32.575351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46e2dc-0ee7-434a-a2c7-d941aab5d340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:32.585816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46e2dc-0ee7-434a-a2c7-d941aab5d340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:32.587268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46e2dc-0ee7-434a-a2c7-d941aab5d340 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:32.597424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 734/1464] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 07:57:34.635600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bc1c1-1a71-4a0f-9da6-4e80e9c6bea8 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:34.636530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bc1c1-1a71-4a0f-9da6-4e80e9c6bea8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:34.636530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bc1c1-1a71-4a0f-9da6-4e80e9c6bea8 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:34.638843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 07:57:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:34.647122 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb362-6514-4949-96aa-3132235a74bb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:34.647589 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb362-6514-4949-96aa-3132235a74bb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:34.647685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb362-6514-4949-96aa-3132235a74bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:34.670301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0c8bb362-6514-4949-96aa-3132235a74bb None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:34.672112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0c8bb362-6514-4949-96aa-3132235a74bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:34.684011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 07:57:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:34.721596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bc1c1-1a71-4a0f-9da6-4e80e9c6bea8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:34.734766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bc1c1-1a71-4a0f-9da6-4e80e9c6bea8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:34.738864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea08db-d9f0-44d5-9ab5-f5f2f39248ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:34.739518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea08db-d9f0-44d5-9ab5-f5f2f39248ac service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:34.739801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea08db-d9f0-44d5-9ab5-f5f2f39248ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:34.740080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea08db-d9f0-44d5-9ab5-f5f2f39248ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:34.750448 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea08db-d9f0-44d5-9ab5-f5f2f39248ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:34.751898 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea08db-d9f0-44d5-9ab5-f5f2f39248ac service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:34.762110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 732/1467] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:36.843814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa230613-d7f7-42d5-b0b3-ce99c2537991 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:36.844453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa230613-d7f7-42d5-b0b3-ce99c2537991 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:36.844569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa230613-d7f7-42d5-b0b3-ce99c2537991 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:36.846883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 07:57:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:36.855681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabc853-711a-4625-9c4d-cd5921ef34ca None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:36.856384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabc853-711a-4625-9c4d-cd5921ef34ca None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:36.856499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabc853-711a-4625-9c4d-cd5921ef34ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:36.875911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-edabc853-711a-4625-9c4d-cd5921ef34ca None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:36.877142 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-edabc853-711a-4625-9c4d-cd5921ef34ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:36.888068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 07:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:36.925707 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa230613-d7f7-42d5-b0b3-ce99c2537991 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:36.942130 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa230613-d7f7-42d5-b0b3-ce99c2537991 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:36.947542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819874e-14f5-4024-b525-c32e89d58afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:36.948460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819874e-14f5-4024-b525-c32e89d58afd service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:36.948802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819874e-14f5-4024-b525-c32e89d58afd service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:36.949097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819874e-14f5-4024-b525-c32e89d58afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:36.964675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819874e-14f5-4024-b525-c32e89d58afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:36.967012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819874e-14f5-4024-b525-c32e89d58afd service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:36.977384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 737/1470] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:46.870449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=11 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:57:46.972607 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=14 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:57:56.872597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f455c85-b336-41c4-9430-1bce8a1f1ca1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:56.873238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f455c85-b336-41c4-9430-1bce8a1f1ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:56.873238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f455c85-b336-41c4-9430-1bce8a1f1ca1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:56.874607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 734/1471] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:56.880649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdafcc-789d-42bf-824c-ff4f018ae4db None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:56.881091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdafcc-789d-42bf-824c-ff4f018ae4db None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:56.881209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdafcc-789d-42bf-824c-ff4f018ae4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:56.899850 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-14cdafcc-789d-42bf-824c-ff4f018ae4db None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:56.901052 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-14cdafcc-789d-42bf-824c-ff4f018ae4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:56.909168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 738/1472] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:56.999504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fb538-6aae-43f3-a03a-34b4a6d3de75 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:56.999802 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fb538-6aae-43f3-a03a-34b4a6d3de75 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:57.000082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fb538-6aae-43f3-a03a-34b4a6d3de75 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:57.001499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 735/1473] 10.222.0.43 () {54 vars in 962 bytes} [Mon Jul 4 07:57:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:57:57.006939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e0ab9-df3b-4579-82ac-c6a2f81ddb08 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:57.007222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e0ab9-df3b-4579-82ac-c6a2f81ddb08 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:57.007498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e0ab9-df3b-4579-82ac-c6a2f81ddb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:57.028262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f72e0ab9-df3b-4579-82ac-c6a2f81ddb08 None None] MFA Rules not processed for user `4a122422a45446c9a63d67b4f931fa47`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:57.029561 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f72e0ab9-df3b-4579-82ac-c6a2f81ddb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:57.037078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 739/1474] 10.222.0.43 () {58 vars in 1059 bytes} [Mon Jul 4 07:57:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:57.044904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fb538-6aae-43f3-a03a-34b4a6d3de75 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:57.057042 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fb538-6aae-43f3-a03a-34b4a6d3de75 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:57.061109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bda80-ed2b-433f-8f85-d763ea8f6947 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a122422a45446c9a63d67b4f931fa47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:57.061802 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bda80-ed2b-433f-8f85-d763ea8f6947 service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:57.062088 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bda80-ed2b-433f-8f85-d763ea8f6947 service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:57.062368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bda80-ed2b-433f-8f85-d763ea8f6947 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:57.072087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bda80-ed2b-433f-8f85-d763ea8f6947 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:57.073544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bda80-ed2b-433f-8f85-d763ea8f6947 service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:57.083177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 736/1475] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:57:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:57:59.197611 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf734-d388-4ae2-a1a4-b3149a4e8f63 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:59.198216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf734-d388-4ae2-a1a4-b3149a4e8f63 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:59.198216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf734-d388-4ae2-a1a4-b3149a4e8f63 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:59.199630 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 740/1476] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:57:59.205571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd997a8-0ddb-4d0a-afaf-ae12ebb205f6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:59.206038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd997a8-0ddb-4d0a-afaf-ae12ebb205f6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:59.206138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd997a8-0ddb-4d0a-afaf-ae12ebb205f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:59.225153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fbd997a8-0ddb-4d0a-afaf-ae12ebb205f6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:57:59.226401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fbd997a8-0ddb-4d0a-afaf-ae12ebb205f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:57:59.235059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 737/1477] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:57:59.259757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cf734-d388-4ae2-a1a4-b3149a4e8f63 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:57:59.274270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cf734-d388-4ae2-a1a4-b3149a4e8f63 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:57:59.279102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae874cb-d2a8-401c-8d77-1d2fa568886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:57:59.279770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae874cb-d2a8-401c-8d77-1d2fa568886f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:57:59.280075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae874cb-d2a8-401c-8d77-1d2fa568886f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:57:59.280353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae874cb-d2a8-401c-8d77-1d2fa568886f admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:57:59.281102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae874cb-d2a8-401c-8d77-1d2fa568886f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:57:59.282957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-3ae874cb-d2a8-401c-8d77-1d2fa568886f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 07:57:59.282957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 04 07:57:59.282957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: Jul 04 07:57:59.284103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae874cb-d2a8-401c-8d77-1d2fa568886f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:57:59.304103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 741/1478] 10.222.0.43 () {56 vars in 1108 bytes} [Mon Jul 4 07:57:59 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 07:58:01.376245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf0e1b-d1e0-424d-ba78-d8da2d89dcb5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:01.376858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf0e1b-d1e0-424d-ba78-d8da2d89dcb5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:01.376858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf0e1b-d1e0-424d-ba78-d8da2d89dcb5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:01.378251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 738/1479] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:58:01.384117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e118aab-a6fa-446d-a3ab-33e8236cd88f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:01.384571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e118aab-a6fa-446d-a3ab-33e8236cd88f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:01.384683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e118aab-a6fa-446d-a3ab-33e8236cd88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:01.404702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4e118aab-a6fa-446d-a3ab-33e8236cd88f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:58:01.405950 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4e118aab-a6fa-446d-a3ab-33e8236cd88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:58:01.415067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 742/1480] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:58:01.459611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf0e1b-d1e0-424d-ba78-d8da2d89dcb5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:58:01.475762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf0e1b-d1e0-424d-ba78-d8da2d89dcb5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:58:01.481873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f020a-7117-45c5-a075-f59e3579fec0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:58:01.482554 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f020a-7117-45c5-a075-f59e3579fec0 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:01.482838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f020a-7117-45c5-a075-f59e3579fec0 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:01.483113 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f020a-7117-45c5-a075-f59e3579fec0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:01.492057 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f020a-7117-45c5-a075-f59e3579fec0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:58:01.493711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f020a-7117-45c5-a075-f59e3579fec0 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:58:01.503980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 739/1481] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:58:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:58:03.801751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f5d3d-7be1-4763-8448-0c74e1a1a611 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:03.802350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f5d3d-7be1-4763-8448-0c74e1a1a611 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:03.802350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f5d3d-7be1-4763-8448-0c74e1a1a611 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:03.803712 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 743/1482] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:58:03.809436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40b714-6a93-4826-ae43-2ece347a6577 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:03.810227 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40b714-6a93-4826-ae43-2ece347a6577 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:03.810227 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40b714-6a93-4826-ae43-2ece347a6577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:03.830444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-aa40b714-6a93-4826-ae43-2ece347a6577 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:58:03.831739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aa40b714-6a93-4826-ae43-2ece347a6577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:58:03.841191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 740/1483] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:58:03.878561 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f5d3d-7be1-4763-8448-0c74e1a1a611 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:58:03.890552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f5d3d-7be1-4763-8448-0c74e1a1a611 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:58:03.894317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43a69b-fd0b-4ecb-817f-dc0d34d65293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:58:03.894956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43a69b-fd0b-4ecb-817f-dc0d34d65293 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:03.895237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43a69b-fd0b-4ecb-817f-dc0d34d65293 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:03.895508 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43a69b-fd0b-4ecb-817f-dc0d34d65293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:03.905611 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43a69b-fd0b-4ecb-817f-dc0d34d65293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:58:03.907102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43a69b-fd0b-4ecb-817f-dc0d34d65293 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:58:03.917009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 744/1484] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:58:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:58:06.097658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cca8d6-3e81-4d96-9262-6af1b2187b77 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:06.098194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cca8d6-3e81-4d96-9262-6af1b2187b77 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:06.098194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cca8d6-3e81-4d96-9262-6af1b2187b77 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:06.099613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 741/1485] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:58:06.105069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff91a9b-dfe6-486f-97a3-026dfe919b5e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:06.105553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff91a9b-dfe6-486f-97a3-026dfe919b5e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:06.105714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff91a9b-dfe6-486f-97a3-026dfe919b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:06.125260 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9ff91a9b-dfe6-486f-97a3-026dfe919b5e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:58:06.126670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9ff91a9b-dfe6-486f-97a3-026dfe919b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:58:06.135132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 745/1486] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:58:06.174934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cca8d6-3e81-4d96-9262-6af1b2187b77 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:58:06.187551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cca8d6-3e81-4d96-9262-6af1b2187b77 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:58:06.191636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb22f2f-9cde-43b0-9338-09cc49d3fbcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:58:06.192292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb22f2f-9cde-43b0-9338-09cc49d3fbcb service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:06.192577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb22f2f-9cde-43b0-9338-09cc49d3fbcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:06.192856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb22f2f-9cde-43b0-9338-09cc49d3fbcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:06.202742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb22f2f-9cde-43b0-9338-09cc49d3fbcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:58:06.204188 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb22f2f-9cde-43b0-9338-09cc49d3fbcb service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:58:06.213776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 742/1487] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:58:08.370416 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f9ab9-f765-4481-9bea-02f15296ca54 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:08.370976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f9ab9-f765-4481-9bea-02f15296ca54 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:08.370976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f9ab9-f765-4481-9bea-02f15296ca54 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:08.372442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 746/1488] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 07:58:08.377977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd591df5-6e97-4c0c-9c9e-8e42283d808c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:08.378435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd591df5-6e97-4c0c-9c9e-8e42283d808c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:08.378542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd591df5-6e97-4c0c-9c9e-8e42283d808c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:08.398370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bd591df5-6e97-4c0c-9c9e-8e42283d808c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:58:08.399605 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bd591df5-6e97-4c0c-9c9e-8e42283d808c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:58:08.409109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 743/1489] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:58:08.449748 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f9ab9-f765-4481-9bea-02f15296ca54 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:58:08.461867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f9ab9-f765-4481-9bea-02f15296ca54 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:58:08.466176 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97beb85a-53d1-4c80-8e7e-74f90f774cb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:58:08.466826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97beb85a-53d1-4c80-8e7e-74f90f774cb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:08.467093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97beb85a-53d1-4c80-8e7e-74f90f774cb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:08.467367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97beb85a-53d1-4c80-8e7e-74f90f774cb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:08.476797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97beb85a-53d1-4c80-8e7e-74f90f774cb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:58:08.478317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97beb85a-53d1-4c80-8e7e-74f90f774cb6 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:58:08.488033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 747/1490] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:58:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:58:10.851941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcc1ef-970e-49e3-bcb0-5b95277b9c10 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:10.852552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcc1ef-970e-49e3-bcb0-5b95277b9c10 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:10.852552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcc1ef-970e-49e3-bcb0-5b95277b9c10 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:10.853929 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 744/1491] 10.222.0.43 () {54 vars in 926 bytes} [Mon Jul 4 07:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:58:10.859419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae81e0-3ac7-4ed1-8b2d-54f60e543833 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:10.859877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae81e0-3ac7-4ed1-8b2d-54f60e543833 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:10.859991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae81e0-3ac7-4ed1-8b2d-54f60e543833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:10.878425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-deae81e0-3ac7-4ed1-8b2d-54f60e543833 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:58:10.880008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-deae81e0-3ac7-4ed1-8b2d-54f60e543833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:58:10.888554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 748/1492] 10.222.0.43 () {58 vars in 1023 bytes} [Mon Jul 4 07:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:58:10.936084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcc1ef-970e-49e3-bcb0-5b95277b9c10 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:58:10.947850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcc1ef-970e-49e3-bcb0-5b95277b9c10 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:58:10.952302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f1b4a-812a-4468-b914-6f0dd06118e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:58:10.952947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f1b4a-812a-4468-b914-6f0dd06118e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:58:10.953227 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f1b4a-812a-4468-b914-6f0dd06118e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:58:10.953505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f1b4a-812a-4468-b914-6f0dd06118e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:58:10.964077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f1b4a-812a-4468-b914-6f0dd06118e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:58:10.966185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f1b4a-812a-4468-b914-6f0dd06118e1 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:58:10.978435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 745/1493] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:58:20.872934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=33 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:58:20.972578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=33 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:59:38.236810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c4043-2b50-4a20-a420-401339dc530b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:38.237470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c4043-2b50-4a20-a420-401339dc530b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:38.237470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c4043-2b50-4a20-a420-401339dc530b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:38.257995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-468c4043-2b50-4a20-a420-401339dc530b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:59:38.259198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-468c4043-2b50-4a20-a420-401339dc530b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:59:38.268998 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 749/1494] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 07:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:59:38.277649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2977c-1580-4f2c-8bfe-c0dda99ace50 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:38.278159 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2977c-1580-4f2c-8bfe-c0dda99ace50 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:38.278287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2977c-1580-4f2c-8bfe-c0dda99ace50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:38.297430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7ce2977c-1580-4f2c-8bfe-c0dda99ace50 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:59:38.298692 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7ce2977c-1580-4f2c-8bfe-c0dda99ace50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:59:38.307941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 746/1495] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 07:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:59:38.315384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c4043-2b50-4a20-a420-401339dc530b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:59:38.326953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c4043-2b50-4a20-a420-401339dc530b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:59:38.332636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b07ff-4a57-4710-8ca1-1fb41d5b5221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:59:38.333329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b07ff-4a57-4710-8ca1-1fb41d5b5221 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:38.333615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b07ff-4a57-4710-8ca1-1fb41d5b5221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:38.333921 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b07ff-4a57-4710-8ca1-1fb41d5b5221 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:38.334658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b07ff-4a57-4710-8ca1-1fb41d5b5221 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 07:59:38.334966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b07ff-4a57-4710-8ca1-1fb41d5b5221 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:59:38.336017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-b50b07ff-4a57-4710-8ca1-1fb41d5b5221 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 07:59:38.336017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 04 07:59:38.336017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: Jul 04 07:59:38.336895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b07ff-4a57-4710-8ca1-1fb41d5b5221 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:59:38.349236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 750/1496] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 07:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 07:59:38.358045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2977c-1580-4f2c-8bfe-c0dda99ace50 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:59:38.367995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2977c-1580-4f2c-8bfe-c0dda99ace50 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:59:38.376233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840657fe-b5c0-4a76-99ad-4d851de1ae22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:59:38.377234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840657fe-b5c0-4a76-99ad-4d851de1ae22 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:38.377670 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840657fe-b5c0-4a76-99ad-4d851de1ae22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:38.378070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840657fe-b5c0-4a76-99ad-4d851de1ae22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:38.379500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840657fe-b5c0-4a76-99ad-4d851de1ae22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:59:38.382329 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-840657fe-b5c0-4a76-99ad-4d851de1ae22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 07:59:38.382329 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Jul 04 07:59:38.382329 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: Jul 04 07:59:38.384149 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840657fe-b5c0-4a76-99ad-4d851de1ae22 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:59:38.431705 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 747/1497] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 07:59:38 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:59:38.436265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b07ff-4a57-4710-8ca1-1fb41d5b5221 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:59:38.447831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b07ff-4a57-4710-8ca1-1fb41d5b5221 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:59:38.509512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f9ee6-7e47-4dd9-95ef-4f788415e99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:59:38.510123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f9ee6-7e47-4dd9-95ef-4f788415e99a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:38.510391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f9ee6-7e47-4dd9-95ef-4f788415e99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:38.510692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f9ee6-7e47-4dd9-95ef-4f788415e99a admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:38.511390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f9ee6-7e47-4dd9-95ef-4f788415e99a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:59:38.512400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-049f9ee6-7e47-4dd9-95ef-4f788415e99a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 07:59:38.512400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 04 07:59:38.512400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: Jul 04 07:59:38.513655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f9ee6-7e47-4dd9-95ef-4f788415e99a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:59:38.540164 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 751/1498] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 07:59:38 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 07:59:38.544847 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840657fe-b5c0-4a76-99ad-4d851de1ae22 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:59:38.555519 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840657fe-b5c0-4a76-99ad-4d851de1ae22 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:59:38.561590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a8339-8da6-4db6-b503-9da1509ec950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:59:38.562301 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a8339-8da6-4db6-b503-9da1509ec950 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:38.562574 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a8339-8da6-4db6-b503-9da1509ec950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:38.562848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a8339-8da6-4db6-b503-9da1509ec950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:38.563582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a8339-8da6-4db6-b503-9da1509ec950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:59:38.564597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-e95a8339-8da6-4db6-b503-9da1509ec950 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 07:59:38.564597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Jul 04 07:59:38.564597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: Jul 04 07:59:38.565525 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a8339-8da6-4db6-b503-9da1509ec950 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:59:38.580940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 748/1499] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 07:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 07:59:38.585917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f9ee6-7e47-4dd9-95ef-4f788415e99a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:59:38.597608 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f9ee6-7e47-4dd9-95ef-4f788415e99a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:59:38.603195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd163a1-2c36-4cc4-a886-3135cca42295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:59:38.604371 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd163a1-2c36-4cc4-a886-3135cca42295 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:38.604656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd163a1-2c36-4cc4-a886-3135cca42295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:38.604938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd163a1-2c36-4cc4-a886-3135cca42295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:38.605711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd163a1-2c36-4cc4-a886-3135cca42295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:59:38.606740 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-acd163a1-2c36-4cc4-a886-3135cca42295 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 07:59:38.606740 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 04 07:59:38.606740 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: Jul 04 07:59:38.607680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd163a1-2c36-4cc4-a886-3135cca42295 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:59:38.622351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 752/1500] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 07:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 07:59:38.626725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a8339-8da6-4db6-b503-9da1509ec950 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:59:38.638901 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a8339-8da6-4db6-b503-9da1509ec950 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:59:38.644058 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca8727-5293-44ad-bf65-1da9d754fcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:59:38.644984 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca8727-5293-44ad-bf65-1da9d754fcc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:38.645271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca8727-5293-44ad-bf65-1da9d754fcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:38.645550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca8727-5293-44ad-bf65-1da9d754fcc0 admin admin] PATH_INFO: `/v3/projects/4e924b92d6fd4e13aaf87b38357bfb2e/users/e9ae606d48cf4a878b7b2957ec535e94/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:38.662459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca8727-5293-44ad-bf65-1da9d754fcc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e924b92d6fd4e13aaf87b38357bfb2e, user_id=e9ae606d48cf4a878b7b2957ec535e94, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:59:38.663530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-43ca8727-5293-44ad-bf65-1da9d754fcc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 07:59:38.663530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Jul 04 07:59:38.663530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: Jul 04 07:59:38.665505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca8727-5293-44ad-bf65-1da9d754fcc0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:59:38.685108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 749/1501] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 07:59:38 2022] PUT /identity/v3/projects/4e924b92d6fd4e13aaf87b38357bfb2e/users/e9ae606d48cf4a878b7b2957ec535e94/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:59:38.698170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c3707-5790-4954-a4aa-28596b98ac37 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:38.698559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c3707-5790-4954-a4aa-28596b98ac37 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:38.698951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c3707-5790-4954-a4aa-28596b98ac37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:38.728102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d18c3707-5790-4954-a4aa-28596b98ac37 None None] MFA Rules not processed for user `e9ae606d48cf4a878b7b2957ec535e94`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:59:38.729429 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d18c3707-5790-4954-a4aa-28596b98ac37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:59:38.803871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 753/1502] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 07:59:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:59:38.817544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1927-00dd-4983-91c3-2ee82301dc6d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:38.817929 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1927-00dd-4983-91c3-2ee82301dc6d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:38.818279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1927-00dd-4983-91c3-2ee82301dc6d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:38.819678 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 750/1503] 10.222.0.43 () {54 vars in 963 bytes} [Mon Jul 4 07:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:59:38.824718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb5219-8505-4e0a-85dd-7f679d7568fa None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:38.825008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb5219-8505-4e0a-85dd-7f679d7568fa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:38.825287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb5219-8505-4e0a-85dd-7f679d7568fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:38.845047 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dddb5219-8505-4e0a-85dd-7f679d7568fa None None] MFA Rules not processed for user `061ffcf242ff4155b92daa8c3888fd10`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:59:38.846352 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dddb5219-8505-4e0a-85dd-7f679d7568fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:59:38.895118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:59:38.905328 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 754/1504] 10.222.0.43 () {58 vars in 1060 bytes} [Mon Jul 4 07:59:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:59:38.918117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1927-00dd-4983-91c3-2ee82301dc6d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:59:38.934657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1927-00dd-4983-91c3-2ee82301dc6d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:59:38.939144 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48d2da-8116-4f53-940f-ff356c048ff0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:59:38.939797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48d2da-8116-4f53-940f-ff356c048ff0 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:38.940082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48d2da-8116-4f53-940f-ff356c048ff0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:38.940352 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48d2da-8116-4f53-940f-ff356c048ff0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:38.950761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48d2da-8116-4f53-940f-ff356c048ff0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:59:38.952234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48d2da-8116-4f53-940f-ff356c048ff0 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:59:38.962334 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 751/1505] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:59:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:59:39.008431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b15df8-8a95-423c-8fb2-cb98b4c8b7c5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:39.008719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b15df8-8a95-423c-8fb2-cb98b4c8b7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:39.009016 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b15df8-8a95-423c-8fb2-cb98b4c8b7c5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:39.010464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 755/1506] 10.222.0.43 () {52 vars in 829 bytes} [Mon Jul 4 07:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 07:59:39.014348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48d2da-8116-4f53-940f-ff356c048ff0 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:59:39.081655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48d2da-8116-4f53-940f-ff356c048ff0 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:59:39.087178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:59:39.088097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:39.088378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:39.088654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] PATH_INFO: `/v3/users/e9ae606d48cf4a878b7b2957ec535e94` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:39.090006 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ae606d48cf4a878b7b2957ec535e94)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:59:39.091040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 07:59:39.091040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Jul 04 07:59:39.091040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: Jul 04 07:59:39.092269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:59:39.170100 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ae606d48cf4a878b7b2957ec535e94'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 07:59:39.170804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ae606d48cf4a878b7b2957ec535e94'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 07:59:39.171591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ae606d48cf4a878b7b2957ec535e94'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 07:59:39.187546 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ae606d48cf4a878b7b2957ec535e94'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 07:59:39.208499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ae606d48cf4a878b7b2957ec535e94'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 07:59:39.212728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:59:39.227257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 752/1507] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 07:59:39 2022] DELETE /identity/v3/users/e9ae606d48cf4a878b7b2957ec535e94 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:59:39.232097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba79fa-45e9-4743-81ce-4dd2a8f43a7a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:39.232384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba79fa-45e9-4743-81ce-4dd2a8f43a7a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:39.232689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba79fa-45e9-4743-81ce-4dd2a8f43a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:39.252713 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-94ba79fa-45e9-4743-81ce-4dd2a8f43a7a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 07:59:39.254287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-94ba79fa-45e9-4743-81ce-4dd2a8f43a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:59:39.329897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 756/1508] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 07:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 07:59:39.340668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:59:39.342378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:59:39.410770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31721e72-334c-4ea3-bbdb-ae1295e15de0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:59:39.415255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892ae3d-be4c-4eff-8ab6-644c91a3a061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:59:39.416049 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892ae3d-be4c-4eff-8ab6-644c91a3a061 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:39.416366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892ae3d-be4c-4eff-8ab6-644c91a3a061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:39.416636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892ae3d-be4c-4eff-8ab6-644c91a3a061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:39.427648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892ae3d-be4c-4eff-8ab6-644c91a3a061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:59:39.429142 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892ae3d-be4c-4eff-8ab6-644c91a3a061 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:59:39.439371 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 753/1509] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 07:59:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 07:59:39.792728 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba79fa-45e9-4743-81ce-4dd2a8f43a7a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 07:59:39.794390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-94ba79fa-45e9-4743-81ce-4dd2a8f43a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 07:59:39.807648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba79fa-45e9-4743-81ce-4dd2a8f43a7a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 07:59:39.813439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 07:59:39.814361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 07:59:39.814642 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 07:59:39.814991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] PATH_INFO: `/v3/projects/4e924b92d6fd4e13aaf87b38357bfb2e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 07:59:39.816336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e924b92d6fd4e13aaf87b38357bfb2e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 07:59:39.817376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 07:59:39.817376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 04 07:59:39.817376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: Jul 04 07:59:39.818687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 07:59:39.881605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e924b92d6fd4e13aaf87b38357bfb2e', 'request_id': 'req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 07:59:39.887835 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e924b92d6fd4e13aaf87b38357bfb2e', 'request_id': 'req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 07:59:39.897793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e924b92d6fd4e13aaf87b38357bfb2e', 'request_id': 'req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 07:59:39.916550 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] The token cache is being invalidate because project 4e924b92d6fd4e13aaf87b38357bfb2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 07:59:39.916838 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 07:59:39.917483 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6cb04280-47b8-4f23-92bf-b8fdc01f24e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 07:59:39.919341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 757/1510] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 07:59:39 2022] DELETE /identity/v3/projects/4e924b92d6fd4e13aaf87b38357bfb2e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 07:59:49.434859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 07:59:49.913211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:01:19.399515 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16905bf8-75b7-417c-91d2-709630c35cfc None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:19.400172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16905bf8-75b7-417c-91d2-709630c35cfc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:19.400172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16905bf8-75b7-417c-91d2-709630c35cfc None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:19.401371 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 08:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:01:19.406720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cab48-798f-4962-89d6-41bee3a4db28 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:19.407178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cab48-798f-4962-89d6-41bee3a4db28 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:19.407295 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cab48-798f-4962-89d6-41bee3a4db28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:19.427296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-840cab48-798f-4962-89d6-41bee3a4db28 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:01:19.428552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-840cab48-798f-4962-89d6-41bee3a4db28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:19.499547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 08:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:01:19.657311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16905bf8-75b7-417c-91d2-709630c35cfc None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:01:19.659547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-16905bf8-75b7-417c-91d2-709630c35cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:19.743502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16905bf8-75b7-417c-91d2-709630c35cfc None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:01:19.750125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c862eb-5686-48fd-b883-7642600392a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:01:19.750798 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c862eb-5686-48fd-b883-7642600392a5 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:19.751080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c862eb-5686-48fd-b883-7642600392a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:19.751362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c862eb-5686-48fd-b883-7642600392a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:19.761015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c862eb-5686-48fd-b883-7642600392a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:01:19.763019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c862eb-5686-48fd-b883-7642600392a5 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:01:19.773044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 755/1513] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:01:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:01:20.516190 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea420e53-2a7f-4f40-8bf1-f7d6366a688f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:20.517011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea420e53-2a7f-4f40-8bf1-f7d6366a688f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:20.517011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea420e53-2a7f-4f40-8bf1-f7d6366a688f None None] PATH_INFO: `/v3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:20.518234 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 759/1514] 10.222.0.61 () {54 vars in 925 bytes} [Mon Jul 4 08:01:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:01:20.525169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c6306-40e2-4a18-b17d-063960c3ffa3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:20.526229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c6306-40e2-4a18-b17d-063960c3ffa3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:20.526229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c6306-40e2-4a18-b17d-063960c3ffa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:20.546311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-520c6306-40e2-4a18-b17d-063960c3ffa3 None None] MFA Rules not processed for user `f57dccfe9f5f4bc397f4c470fbf047dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:01:20.547568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-520c6306-40e2-4a18-b17d-063960c3ffa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:20.615323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad7e06-12a6-45ed-97a0-a915e69ea5dd None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:20.615612 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad7e06-12a6-45ed-97a0-a915e69ea5dd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:20.615900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad7e06-12a6-45ed-97a0-a915e69ea5dd None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:20.617450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 760/1515] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 08:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:01:20.622516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b430f0-27f0-4fc0-bee2-13053911e043 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:20.622802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b430f0-27f0-4fc0-bee2-13053911e043 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:20.623076 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b430f0-27f0-4fc0-bee2-13053911e043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:20.624820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 756/1516] 10.222.0.61 () {58 vars in 1015 bytes} [Mon Jul 4 08:01:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:01:20.644659 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-68b430f0-27f0-4fc0-bee2-13053911e043 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:01:20.645935 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-68b430f0-27f0-4fc0-bee2-13053911e043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:20.656112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 761/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 08:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:01:20.855930 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c6306-40e2-4a18-b17d-063960c3ffa3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:01:20.869556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c6306-40e2-4a18-b17d-063960c3ffa3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:01:20.877063 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffffb4a-3f8d-4edb-a885-6a770209aec0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:01:20.878146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffffb4a-3f8d-4edb-a885-6a770209aec0 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:20.878433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffffb4a-3f8d-4edb-a885-6a770209aec0 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:20.878711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffffb4a-3f8d-4edb-a885-6a770209aec0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:20.888163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffffb4a-3f8d-4edb-a885-6a770209aec0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:01:20.889812 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffffb4a-3f8d-4edb-a885-6a770209aec0 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:01:20.901273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 757/1518] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:01:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:01:29.894604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc15ad3-f5ab-4346-bf35-20e97a78c768 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:29.895477 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc15ad3-f5ab-4346-bf35-20e97a78c768 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:29.895477 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc15ad3-f5ab-4346-bf35-20e97a78c768 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:29.897433 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 762/1519] 10.222.0.43 () {54 vars in 960 bytes} [Mon Jul 4 08:01:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:01:29.904461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0613f-e180-4ed4-946e-180731e14077 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:29.904943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0613f-e180-4ed4-946e-180731e14077 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:29.905047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0613f-e180-4ed4-946e-180731e14077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:29.925397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-90e0613f-e180-4ed4-946e-180731e14077 None None] MFA Rules not processed for user `f57dccfe9f5f4bc397f4c470fbf047dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:01:29.926661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-90e0613f-e180-4ed4-946e-180731e14077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:29.936946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 758/1520] 10.222.0.43 () {58 vars in 1057 bytes} [Mon Jul 4 08:01:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:01:29.944952 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc15ad3-f5ab-4346-bf35-20e97a78c768 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:01:29.961958 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc15ad3-f5ab-4346-bf35-20e97a78c768 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:01:29.971086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e6782-46f9-4057-bdc3-320566443d24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57dccfe9f5f4bc397f4c470fbf047dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:01:29.971953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e6782-46f9-4057-bdc3-320566443d24 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:29.972268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e6782-46f9-4057-bdc3-320566443d24 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:29.972540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e6782-46f9-4057-bdc3-320566443d24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:29.982832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e6782-46f9-4057-bdc3-320566443d24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:01:29.984461 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e6782-46f9-4057-bdc3-320566443d24 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:01:29.995079 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 763/1521] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:01:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:01:32.185212 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1d113-97b0-4894-af6a-c0eb79aa83e5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:32.186146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1d113-97b0-4894-af6a-c0eb79aa83e5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:32.186146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1d113-97b0-4894-af6a-c0eb79aa83e5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:32.188265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 08:01:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:01:32.196338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27875b5-b331-4d42-a46b-0d065bbc2f3a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:32.196819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27875b5-b331-4d42-a46b-0d065bbc2f3a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:32.196923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27875b5-b331-4d42-a46b-0d065bbc2f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:32.216294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a27875b5-b331-4d42-a46b-0d065bbc2f3a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:01:32.217510 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a27875b5-b331-4d42-a46b-0d065bbc2f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:32.226979 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 08:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:01:32.385165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1d113-97b0-4894-af6a-c0eb79aa83e5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:01:32.401324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1d113-97b0-4894-af6a-c0eb79aa83e5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:01:32.407710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cac11-30b5-457b-a8db-043c3f977ef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:01:32.408366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cac11-30b5-457b-a8db-043c3f977ef5 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:32.408645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cac11-30b5-457b-a8db-043c3f977ef5 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:32.408890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cac11-30b5-457b-a8db-043c3f977ef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:32.418081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cac11-30b5-457b-a8db-043c3f977ef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:01:32.419729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cac11-30b5-457b-a8db-043c3f977ef5 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:01:32.429590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 760/1524] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:01:33.274454 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d51df5-add4-4e5f-ab75-fa552968527d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:33.275364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d51df5-add4-4e5f-ab75-fa552968527d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:33.275364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d51df5-add4-4e5f-ab75-fa552968527d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:33.278198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 08:01:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:01:33.285575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf01244-24a7-4142-9bda-fb51b4fa1a97 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:33.286099 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf01244-24a7-4142-9bda-fb51b4fa1a97 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:33.286198 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf01244-24a7-4142-9bda-fb51b4fa1a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:33.306583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-daf01244-24a7-4142-9bda-fb51b4fa1a97 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:01:33.308313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-daf01244-24a7-4142-9bda-fb51b4fa1a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:33.317229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 08:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:01:33.476558 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d51df5-add4-4e5f-ab75-fa552968527d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:01:33.500497 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d51df5-add4-4e5f-ab75-fa552968527d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:01:33.507399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9b5b4-c19a-4a88-9ce3-21918da4227b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:01:33.508049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9b5b4-c19a-4a88-9ce3-21918da4227b service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:33.508350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9b5b4-c19a-4a88-9ce3-21918da4227b service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:33.508628 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9b5b4-c19a-4a88-9ce3-21918da4227b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:33.517744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9b5b4-c19a-4a88-9ce3-21918da4227b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:01:33.519411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9b5b4-c19a-4a88-9ce3-21918da4227b service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:01:33.530101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 766/1527] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:01:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:01:37.508858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7f064-752b-48b4-b4fc-44e0ef7e665e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:37.509791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7f064-752b-48b4-b4fc-44e0ef7e665e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:37.509791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7f064-752b-48b4-b4fc-44e0ef7e665e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:37.511332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 08:01:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:01:37.516971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c8b10-4ddf-418f-8c10-492a1030a920 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:37.517454 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c8b10-4ddf-418f-8c10-492a1030a920 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:37.517563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c8b10-4ddf-418f-8c10-492a1030a920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:37.540390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ad9c8b10-4ddf-418f-8c10-492a1030a920 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:01:37.542079 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ad9c8b10-4ddf-418f-8c10-492a1030a920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:37.551481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 08:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:01:37.630276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7f064-752b-48b4-b4fc-44e0ef7e665e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:01:37.633058 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-69d7f064-752b-48b4-b4fc-44e0ef7e665e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:37.681893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:01:37.694467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7f064-752b-48b4-b4fc-44e0ef7e665e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:01:37.698550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792dd55-8533-4ecb-b6aa-7c43a56030f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:01:37.699196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792dd55-8533-4ecb-b6aa-7c43a56030f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:37.699483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792dd55-8533-4ecb-b6aa-7c43a56030f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:37.699762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792dd55-8533-4ecb-b6aa-7c43a56030f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:37.709171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792dd55-8533-4ecb-b6aa-7c43a56030f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:01:37.710673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792dd55-8533-4ecb-b6aa-7c43a56030f5 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:01:37.720187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 763/1530] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:01:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:01:39.748719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e98b3-c117-4c2e-9f95-d92aba31a313 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:39.749575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e98b3-c117-4c2e-9f95-d92aba31a313 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:39.749575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e98b3-c117-4c2e-9f95-d92aba31a313 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:39.751948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 08:01:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:01:39.761394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06367cc3-0307-442f-a9f9-2a4bd8253571 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:39.761902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06367cc3-0307-442f-a9f9-2a4bd8253571 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:39.762061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06367cc3-0307-442f-a9f9-2a4bd8253571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:39.790435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-06367cc3-0307-442f-a9f9-2a4bd8253571 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:01:39.792083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-06367cc3-0307-442f-a9f9-2a4bd8253571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:39.805503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 08:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:01:39.844661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e98b3-c117-4c2e-9f95-d92aba31a313 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:01:39.846748 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-270e98b3-c117-4c2e-9f95-d92aba31a313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:39.861984 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e98b3-c117-4c2e-9f95-d92aba31a313 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:01:39.866049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4449b-cbb2-4375-b14f-57be61e80dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:01:39.866695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4449b-cbb2-4375-b14f-57be61e80dad service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:39.866983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4449b-cbb2-4375-b14f-57be61e80dad service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:39.867273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4449b-cbb2-4375-b14f-57be61e80dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:39.876848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4449b-cbb2-4375-b14f-57be61e80dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:01:39.878671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4449b-cbb2-4375-b14f-57be61e80dad service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:01:39.888720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 769/1533] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:01:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:01:43.588548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70eac8-3c75-4944-b99d-844196d5d1f9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:43.589159 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70eac8-3c75-4944-b99d-844196d5d1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:43.589159 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70eac8-3c75-4944-b99d-844196d5d1f9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:43.590521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 08:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:01:43.597106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10f8d7-3aa5-4819-bb77-fd12c15bd979 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:43.597873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10f8d7-3aa5-4819-bb77-fd12c15bd979 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:43.598174 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10f8d7-3aa5-4819-bb77-fd12c15bd979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:43.608576 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=55 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:01:43.621865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9c10f8d7-3aa5-4819-bb77-fd12c15bd979 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:01:43.623107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9c10f8d7-3aa5-4819-bb77-fd12c15bd979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:43.632546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 08:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:01:43.935522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70eac8-3c75-4944-b99d-844196d5d1f9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:01:43.937399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ed70eac8-3c75-4944-b99d-844196d5d1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:44.003983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70eac8-3c75-4944-b99d-844196d5d1f9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:01:44.007947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f00ad4-8c3e-4093-bcf8-bfa8431548d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:01:44.008596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f00ad4-8c3e-4093-bcf8-bfa8431548d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:44.008881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f00ad4-8c3e-4093-bcf8-bfa8431548d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:44.009131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f00ad4-8c3e-4093-bcf8-bfa8431548d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:44.018671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f00ad4-8c3e-4093-bcf8-bfa8431548d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:01:44.020105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f00ad4-8c3e-4093-bcf8-bfa8431548d8 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:01:44.029869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 766/1536] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:01:46.135599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af34a7-a7dc-4362-af95-236e9ff795a8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:46.136283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af34a7-a7dc-4362-af95-236e9ff795a8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:46.136424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af34a7-a7dc-4362-af95-236e9ff795a8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:46.137855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 08:01:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:01:46.146295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de03073-0e66-465a-863c-5f500bcfde9d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:46.146755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de03073-0e66-465a-863c-5f500bcfde9d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:46.146885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de03073-0e66-465a-863c-5f500bcfde9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:46.167537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2de03073-0e66-465a-863c-5f500bcfde9d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:01:46.168759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2de03073-0e66-465a-863c-5f500bcfde9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:46.178958 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 08:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:01:46.484971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af34a7-a7dc-4362-af95-236e9ff795a8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:01:46.487710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-22af34a7-a7dc-4362-af95-236e9ff795a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:01:46.503889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af34a7-a7dc-4362-af95-236e9ff795a8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:01:46.508145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0e93d-2657-46b3-ad40-b77785b39703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:01:46.508787 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0e93d-2657-46b3-ad40-b77785b39703 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:01:46.509066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0e93d-2657-46b3-ad40-b77785b39703 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:01:46.509337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0e93d-2657-46b3-ad40-b77785b39703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:01:46.519332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0e93d-2657-46b3-ad40-b77785b39703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:01:46.520766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0e93d-2657-46b3-ad40-b77785b39703 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:01:46.530447 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 772/1539] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:01:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:01:56.161871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=39 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:01:56.526367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=8 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:19.168200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9eaa4-726c-4956-9c00-69d01779f0d3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:19.170607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9eaa4-726c-4956-9c00-69d01779f0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:19.171629 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9eaa4-726c-4956-9c00-69d01779f0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:19.217228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-93c9eaa4-726c-4956-9c00-69d01779f0d3 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:19.220286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-93c9eaa4-726c-4956-9c00-69d01779f0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:19.241050 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 768/1540] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:19.277408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8834a-541c-4ccc-b35f-e5f4d7400162 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:19.278376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8834a-541c-4ccc-b35f-e5f4d7400162 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:19.278376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8834a-541c-4ccc-b35f-e5f4d7400162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:19.306969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2cf8834a-541c-4ccc-b35f-e5f4d7400162 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:19.308247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2cf8834a-541c-4ccc-b35f-e5f4d7400162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:19.321793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 773/1541] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:19.328686 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9eaa4-726c-4956-9c00-69d01779f0d3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:19.344457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9eaa4-726c-4956-9c00-69d01779f0d3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:19.353431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811703e-972d-4f0c-88a1-7f172ecffe85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:19.354187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811703e-972d-4f0c-88a1-7f172ecffe85 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:19.354463 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811703e-972d-4f0c-88a1-7f172ecffe85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:19.354722 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811703e-972d-4f0c-88a1-7f172ecffe85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:19.355611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811703e-972d-4f0c-88a1-7f172ecffe85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:03:19.356007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811703e-972d-4f0c-88a1-7f172ecffe85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:19.357235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-0811703e-972d-4f0c-88a1-7f172ecffe85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 08:03:19.357235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Jul 04 08:03:19.357235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: Jul 04 08:03:19.358269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811703e-972d-4f0c-88a1-7f172ecffe85 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:19.380689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 769/1542] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:03:19.397095 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8834a-541c-4ccc-b35f-e5f4d7400162 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:19.421183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8834a-541c-4ccc-b35f-e5f4d7400162 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:19.438649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e04bea-3a26-440b-bcc2-82a3ca34b014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:19.440453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e04bea-3a26-440b-bcc2-82a3ca34b014 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:19.441245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e04bea-3a26-440b-bcc2-82a3ca34b014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:19.442023 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e04bea-3a26-440b-bcc2-82a3ca34b014 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:19.444945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e04bea-3a26-440b-bcc2-82a3ca34b014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:19.447460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-83e04bea-3a26-440b-bcc2-82a3ca34b014 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 08:03:19.447460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 04 08:03:19.447460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: Jul 04 08:03:19.450345 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e04bea-3a26-440b-bcc2-82a3ca34b014 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:19.526193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 774/1543] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:19 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:03:19.531705 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811703e-972d-4f0c-88a1-7f172ecffe85 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:19.546622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811703e-972d-4f0c-88a1-7f172ecffe85 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:19.644018 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735edd93-22aa-456d-a2b7-6406ab20306f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:19.646547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735edd93-22aa-456d-a2b7-6406ab20306f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:19.650829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735edd93-22aa-456d-a2b7-6406ab20306f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:19.738646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-735edd93-22aa-456d-a2b7-6406ab20306f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:19.741545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-735edd93-22aa-456d-a2b7-6406ab20306f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:19.783286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1edba-a268-495b-a039-c83ae85b92f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:19.784005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1edba-a268-495b-a039-c83ae85b92f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:19.784302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1edba-a268-495b-a039-c83ae85b92f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:19.784578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1edba-a268-495b-a039-c83ae85b92f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:19.785290 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1edba-a268-495b-a039-c83ae85b92f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:19.787083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-d8f1edba-a268-495b-a039-c83ae85b92f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 08:03:19.787083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Jul 04 08:03:19.787083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: Jul 04 08:03:19.788667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1edba-a268-495b-a039-c83ae85b92f8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:19.842362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 770/1544] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:19 2022] POST /identity/v3/users => generated 289 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:19.850656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1eb3b6-5c96-4ca2-b8cf-14cca7c399e0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:19.851184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1eb3b6-5c96-4ca2-b8cf-14cca7c399e0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:19.851697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1eb3b6-5c96-4ca2-b8cf-14cca7c399e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:19.888016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6b1eb3b6-5c96-4ca2-b8cf-14cca7c399e0 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:19.889319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6b1eb3b6-5c96-4ca2-b8cf-14cca7c399e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:19.899204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 771/1545] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:19.900703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d670e05-06c1-42fd-9ce0-8e8dec63bf21 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:19.900995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d670e05-06c1-42fd-9ce0-8e8dec63bf21 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:19.901278 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d670e05-06c1-42fd-9ce0-8e8dec63bf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:19.911714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 775/1546] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:19.911714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717fcca-6fe4-431e-9c43-924fa8c86bf0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:19.911714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717fcca-6fe4-431e-9c43-924fa8c86bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:19.911714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717fcca-6fe4-431e-9c43-924fa8c86bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:19.929286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d717fcca-6fe4-431e-9c43-924fa8c86bf0 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:19.930574 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d717fcca-6fe4-431e-9c43-924fa8c86bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:19.936452 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0d670e05-06c1-42fd-9ce0-8e8dec63bf21 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:19.938487 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0d670e05-06c1-42fd-9ce0-8e8dec63bf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:19.938920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 776/1547] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:19.940359 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b371a4-f124-4913-ac24-7b5551afdcab None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:19.940683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b371a4-f124-4913-ac24-7b5551afdcab None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:19.940916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b371a4-f124-4913-ac24-7b5551afdcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:19.952782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 772/1548] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:19.954277 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d670e05-06c1-42fd-9ce0-8e8dec63bf21 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:19.971323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-13b371a4-f124-4913-ac24-7b5551afdcab None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:19.979926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d670e05-06c1-42fd-9ce0-8e8dec63bf21 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:19.981825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-13b371a4-f124-4913-ac24-7b5551afdcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:19.982095 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9315586-f0a5-464a-a311-af065563d28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:19.982808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9315586-f0a5-464a-a311-af065563d28b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:19.983106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9315586-f0a5-464a-a311-af065563d28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:19.983355 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9315586-f0a5-464a-a311-af065563d28b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:19.984113 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9315586-f0a5-464a-a311-af065563d28b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:19.984886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 777/1549] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:19.986018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9315586-f0a5-464a-a311-af065563d28b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:19.993527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ff98b-93e9-4a6d-91e8-9a4946d7c1ff None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:19.993527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ff98b-93e9-4a6d-91e8-9a4946d7c1ff None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:19.994090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ff98b-93e9-4a6d-91e8-9a4946d7c1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:19.998755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 773/1550] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:20.000209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7313b5-27a7-4d56-b889-d589c4f8502e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.000496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7313b5-27a7-4d56-b889-d589c4f8502e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.000768 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7313b5-27a7-4d56-b889-d589c4f8502e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.026624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-987ff98b-93e9-4a6d-91e8-9a4946d7c1ff None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:20.028569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-987ff98b-93e9-4a6d-91e8-9a4946d7c1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:20.028904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ab7313b5-27a7-4d56-b889-d589c4f8502e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:20.030813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ab7313b5-27a7-4d56-b889-d589c4f8502e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:20.042723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 774/1551] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:20.044042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 778/1552] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:20.052660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4727d-c818-4c66-9f71-0384bed64d05 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.052660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4727d-c818-4c66-9f71-0384bed64d05 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.052660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4727d-c818-4c66-9f71-0384bed64d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.053216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfe850-d30d-4a6c-ae44-0e801273b22e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.053216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfe850-d30d-4a6c-ae44-0e801273b22e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.053216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfe850-d30d-4a6c-ae44-0e801273b22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.072662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bac4727d-c818-4c66-9f71-0384bed64d05 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:20.073529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6dcfe850-d30d-4a6c-ae44-0e801273b22e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:20.074329 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bac4727d-c818-4c66-9f71-0384bed64d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:20.074974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6dcfe850-d30d-4a6c-ae44-0e801273b22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:20.084616 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 779/1553] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:20.085911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 775/1554] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:20.086362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8866fca-226d-418b-8c91-ad583ecdbabd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.086836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8866fca-226d-418b-8c91-ad583ecdbabd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.087308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8866fca-226d-418b-8c91-ad583ecdbabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.092712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951aa5e6-af48-4314-9c3b-a6d0b3d9eeda None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.093844 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951aa5e6-af48-4314-9c3b-a6d0b3d9eeda None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.094355 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951aa5e6-af48-4314-9c3b-a6d0b3d9eeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.118421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d8866fca-226d-418b-8c91-ad583ecdbabd None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:20.120566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d8866fca-226d-418b-8c91-ad583ecdbabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:20.121155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-951aa5e6-af48-4314-9c3b-a6d0b3d9eeda None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:20.122467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-951aa5e6-af48-4314-9c3b-a6d0b3d9eeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:20.131808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 776/1555] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:20.133206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feca62e-f186-4cfe-b8cc-cf00e6f34e4e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.133514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feca62e-f186-4cfe-b8cc-cf00e6f34e4e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.133841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feca62e-f186-4cfe-b8cc-cf00e6f34e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.134525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 780/1556] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:20.134525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8866fca-226d-418b-8c91-ad583ecdbabd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.154468 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8866fca-226d-418b-8c91-ad583ecdbabd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.159582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8feca62e-f186-4cfe-b8cc-cf00e6f34e4e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:20.160905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8feca62e-f186-4cfe-b8cc-cf00e6f34e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:20.164648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca66b6-44e6-4d7f-ab2d-73030f846a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.165861 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca66b6-44e6-4d7f-ab2d-73030f846a5a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.166364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca66b6-44e6-4d7f-ab2d-73030f846a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.166848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca66b6-44e6-4d7f-ab2d-73030f846a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.168282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca66b6-44e6-4d7f-ab2d-73030f846a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.171813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 777/1557] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:20.172674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97138183-40c3-4780-ace6-2d5ed168506b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.172674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97138183-40c3-4780-ace6-2d5ed168506b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.172969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97138183-40c3-4780-ace6-2d5ed168506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.177578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca66b6-44e6-4d7f-ab2d-73030f846a5a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.191017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 781/1558] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:20.192532 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5931443-c2b0-4c65-83bb-e9bb7dc59c1e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.192887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5931443-c2b0-4c65-83bb-e9bb7dc59c1e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.193263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5931443-c2b0-4c65-83bb-e9bb7dc59c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.205075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-97138183-40c3-4780-ace6-2d5ed168506b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:20.207277 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-97138183-40c3-4780-ace6-2d5ed168506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:20.214860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e5931443-c2b0-4c65-83bb-e9bb7dc59c1e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:20.216887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e5931443-c2b0-4c65-83bb-e9bb7dc59c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:20.225363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 778/1559] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:20.225363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97138183-40c3-4780-ace6-2d5ed168506b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.231197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 782/1560] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:20.233406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5931443-c2b0-4c65-83bb-e9bb7dc59c1e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.245271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5931443-c2b0-4c65-83bb-e9bb7dc59c1e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.250047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97138183-40c3-4780-ace6-2d5ed168506b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.251978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe81ad9-a4bc-492c-a551-48498a08dc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.253033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe81ad9-a4bc-492c-a551-48498a08dc5b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.253304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe81ad9-a4bc-492c-a551-48498a08dc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.253567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe81ad9-a4bc-492c-a551-48498a08dc5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.254379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe81ad9-a4bc-492c-a551-48498a08dc5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:03:20.254683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe81ad9-a4bc-492c-a551-48498a08dc5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.256398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe81ad9-a4bc-492c-a551-48498a08dc5b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.258650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099cacf-93fa-4088-b61f-b6869783d7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.259793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099cacf-93fa-4088-b61f-b6869783d7b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.260235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099cacf-93fa-4088-b61f-b6869783d7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.260629 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099cacf-93fa-4088-b61f-b6869783d7b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.261870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099cacf-93fa-4088-b61f-b6869783d7b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:03:20.262413 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099cacf-93fa-4088-b61f-b6869783d7b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.265426 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099cacf-93fa-4088-b61f-b6869783d7b8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.270875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 783/1561] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:03:20.271228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe81ad9-a4bc-492c-a551-48498a08dc5b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.291025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 779/1562] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:03:20.291298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe81ad9-a4bc-492c-a551-48498a08dc5b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.292517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099cacf-93fa-4088-b61f-b6869783d7b8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.299558 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764044dd-eb0f-46b7-ab77-a4cee5d9e5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.300690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764044dd-eb0f-46b7-ab77-a4cee5d9e5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.301202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764044dd-eb0f-46b7-ab77-a4cee5d9e5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.301831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764044dd-eb0f-46b7-ab77-a4cee5d9e5b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.303215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764044dd-eb0f-46b7-ab77-a4cee5d9e5b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:03:20.303750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764044dd-eb0f-46b7-ab77-a4cee5d9e5b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.306823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764044dd-eb0f-46b7-ab77-a4cee5d9e5b3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.310675 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099cacf-93fa-4088-b61f-b6869783d7b8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.318832 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61101388-b3b8-4afe-bb15-edfa5fa4e90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.320022 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61101388-b3b8-4afe-bb15-edfa5fa4e90a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.320517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61101388-b3b8-4afe-bb15-edfa5fa4e90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.320974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61101388-b3b8-4afe-bb15-edfa5fa4e90a admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.322428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61101388-b3b8-4afe-bb15-edfa5fa4e90a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:03:20.322987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61101388-b3b8-4afe-bb15-edfa5fa4e90a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.326141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61101388-b3b8-4afe-bb15-edfa5fa4e90a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.335003 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 784/1563] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:03:20.336110 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764044dd-eb0f-46b7-ab77-a4cee5d9e5b3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.346904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 780/1564] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:03:20.347830 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61101388-b3b8-4afe-bb15-edfa5fa4e90a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.355044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764044dd-eb0f-46b7-ab77-a4cee5d9e5b3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.362460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61101388-b3b8-4afe-bb15-edfa5fa4e90a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.362843 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6ed69-84ad-4dad-a006-dd6b79499a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.363538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6ed69-84ad-4dad-a006-dd6b79499a57 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.364040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6ed69-84ad-4dad-a006-dd6b79499a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.365024 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6ed69-84ad-4dad-a006-dd6b79499a57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.366690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6ed69-84ad-4dad-a006-dd6b79499a57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:03:20.367206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6ed69-84ad-4dad-a006-dd6b79499a57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.369192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca9050-8046-43ec-b9cc-663071bd5734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.370160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca9050-8046-43ec-b9cc-663071bd5734 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.370331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6ed69-84ad-4dad-a006-dd6b79499a57 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.370470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca9050-8046-43ec-b9cc-663071bd5734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.370710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca9050-8046-43ec-b9cc-663071bd5734 admin admin] PATH_INFO: `/v3/projects/34a4b394ba684ed2a67d14b4a5b24f2f/users/18b1072539af49729dff7ee9ea445c77/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.388881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca9050-8046-43ec-b9cc-663071bd5734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a4b394ba684ed2a67d14b4a5b24f2f, user_id=18b1072539af49729dff7ee9ea445c77, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.391714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca9050-8046-43ec-b9cc-663071bd5734 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.393414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 785/1565] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:03:20.394816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6ed69-84ad-4dad-a006-dd6b79499a57 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.415665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 781/1566] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:20 2022] PUT /identity/v3/projects/34a4b394ba684ed2a67d14b4a5b24f2f/users/18b1072539af49729dff7ee9ea445c77/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:20.416823 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca9050-8046-43ec-b9cc-663071bd5734 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.425862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6ed69-84ad-4dad-a006-dd6b79499a57 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.448570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:20.496499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:20.501606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca9050-8046-43ec-b9cc-663071bd5734 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.508922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5860bd-220c-434e-ab61-a49ad6749916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.510013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5860bd-220c-434e-ab61-a49ad6749916 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.510496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5860bd-220c-434e-ab61-a49ad6749916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.510928 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5860bd-220c-434e-ab61-a49ad6749916 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.512204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5860bd-220c-434e-ab61-a49ad6749916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:03:20.512630 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5860bd-220c-434e-ab61-a49ad6749916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.514573 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5860bd-220c-434e-ab61-a49ad6749916 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.526046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f5b24-ce92-41af-afb1-5cc80ae09d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.526752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f5b24-ce92-41af-afb1-5cc80ae09d1c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.527038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f5b24-ce92-41af-afb1-5cc80ae09d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.527314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f5b24-ce92-41af-afb1-5cc80ae09d1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.528078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f5b24-ce92-41af-afb1-5cc80ae09d1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:03:20.528525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f5b24-ce92-41af-afb1-5cc80ae09d1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.530179 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f5b24-ce92-41af-afb1-5cc80ae09d1c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.539852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 782/1567] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:03:20.539852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5860bd-220c-434e-ab61-a49ad6749916 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.551422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 786/1568] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:03:20.552651 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5860bd-220c-434e-ab61-a49ad6749916 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.553190 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f5b24-ce92-41af-afb1-5cc80ae09d1c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.562529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2be972-ea70-40af-b922-e77add99ec42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.563656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2be972-ea70-40af-b922-e77add99ec42 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.564157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2be972-ea70-40af-b922-e77add99ec42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.564599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2be972-ea70-40af-b922-e77add99ec42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.565852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2be972-ea70-40af-b922-e77add99ec42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.568924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2be972-ea70-40af-b922-e77add99ec42 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.573325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f5b24-ce92-41af-afb1-5cc80ae09d1c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.580850 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb011e42-60bd-4328-ab19-9eef80cc7a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.581975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb011e42-60bd-4328-ab19-9eef80cc7a12 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.582430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb011e42-60bd-4328-ab19-9eef80cc7a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.582889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb011e42-60bd-4328-ab19-9eef80cc7a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.584087 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb011e42-60bd-4328-ab19-9eef80cc7a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.587097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb011e42-60bd-4328-ab19-9eef80cc7a12 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.610884 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 783/1569] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:20.611920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2be972-ea70-40af-b922-e77add99ec42 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.630361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 787/1570] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/projects => generated 379 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:20.632205 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb011e42-60bd-4328-ab19-9eef80cc7a12 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.692668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2be972-ea70-40af-b922-e77add99ec42 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.742057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb011e42-60bd-4328-ab19-9eef80cc7a12 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.747284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b0fe7-fb48-4121-aa3a-1279534b6409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.747971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b0fe7-fb48-4121-aa3a-1279534b6409 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.748252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b0fe7-fb48-4121-aa3a-1279534b6409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.748525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b0fe7-fb48-4121-aa3a-1279534b6409 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.749236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b0fe7-fb48-4121-aa3a-1279534b6409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.751317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b0fe7-fb48-4121-aa3a-1279534b6409 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.777879 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6712c-e799-42ab-a7c0-e9a70cec5904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.779048 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6712c-e799-42ab-a7c0-e9a70cec5904 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.779564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6712c-e799-42ab-a7c0-e9a70cec5904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.780065 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6712c-e799-42ab-a7c0-e9a70cec5904 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.781446 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6712c-e799-42ab-a7c0-e9a70cec5904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.785382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6712c-e799-42ab-a7c0-e9a70cec5904 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.785994 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 788/1571] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:20.788255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b0fe7-fb48-4121-aa3a-1279534b6409 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.830237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 784/1572] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/projects => generated 385 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:20.835450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4f72d-92a7-47f7-a568-553bb33531fe None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.835737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4f72d-92a7-47f7-a568-553bb33531fe None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.836023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4f72d-92a7-47f7-a568-553bb33531fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.859600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-10b4f72d-92a7-47f7-a568-553bb33531fe None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:20.861859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-10b4f72d-92a7-47f7-a568-553bb33531fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:20.881875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b0fe7-fb48-4121-aa3a-1279534b6409 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.900234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:20.943056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:20.967494 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 785/1573] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:20.969157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4f72d-92a7-47f7-a568-553bb33531fe None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:20.971625 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609574fd-9d75-470d-b38c-d561cd613b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.973262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609574fd-9d75-470d-b38c-d561cd613b05 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.973262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609574fd-9d75-470d-b38c-d561cd613b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.973262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609574fd-9d75-470d-b38c-d561cd613b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.974087 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609574fd-9d75-470d-b38c-d561cd613b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:20.976487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609574fd-9d75-470d-b38c-d561cd613b05 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:20.988737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4f72d-92a7-47f7-a568-553bb33531fe None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:20.994795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118763c6-38e1-4c74-8334-831ec443f078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:20.995752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118763c6-38e1-4c74-8334-831ec443f078 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:20.996132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118763c6-38e1-4c74-8334-831ec443f078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:20.996538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118763c6-38e1-4c74-8334-831ec443f078 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:20.997350 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118763c6-38e1-4c74-8334-831ec443f078 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.000170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118763c6-38e1-4c74-8334-831ec443f078 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.019285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 789/1574] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:21.021395 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609574fd-9d75-470d-b38c-d561cd613b05 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.037053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 786/1575] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:20 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:21.038210 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118763c6-38e1-4c74-8334-831ec443f078 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.114670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609574fd-9d75-470d-b38c-d561cd613b05 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.145863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118763c6-38e1-4c74-8334-831ec443f078 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.154333 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0075083-f945-4104-aeb3-d05cdb01eecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.155528 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0075083-f945-4104-aeb3-d05cdb01eecf admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.156033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0075083-f945-4104-aeb3-d05cdb01eecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.156516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0075083-f945-4104-aeb3-d05cdb01eecf admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.157838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0075083-f945-4104-aeb3-d05cdb01eecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.161709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0075083-f945-4104-aeb3-d05cdb01eecf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.186651 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 787/1576] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:21 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:21.187598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0075083-f945-4104-aeb3-d05cdb01eecf admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.198772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0075083-f945-4104-aeb3-d05cdb01eecf admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.204374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5293b0-1aef-43bc-8330-434397a509cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.205101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5293b0-1aef-43bc-8330-434397a509cb admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.205387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5293b0-1aef-43bc-8330-434397a509cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.205700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5293b0-1aef-43bc-8330-434397a509cb admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.206429 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5293b0-1aef-43bc-8330-434397a509cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.208515 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5293b0-1aef-43bc-8330-434397a509cb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.216203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568cd05-4f94-4984-9fb3-299b00b156ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.217395 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568cd05-4f94-4984-9fb3-299b00b156ff admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.217947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568cd05-4f94-4984-9fb3-299b00b156ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.218439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568cd05-4f94-4984-9fb3-299b00b156ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.219764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568cd05-4f94-4984-9fb3-299b00b156ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.224433 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568cd05-4f94-4984-9fb3-299b00b156ff admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.232337 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 788/1577] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:21 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:21.233208 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5293b0-1aef-43bc-8330-434397a509cb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.244412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5293b0-1aef-43bc-8330-434397a509cb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.250595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bbd37-0454-4747-864e-829990ea22f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.251359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bbd37-0454-4747-864e-829990ea22f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.251635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bbd37-0454-4747-864e-829990ea22f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.251981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bbd37-0454-4747-864e-829990ea22f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.253070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015bbd37-0454-4747-864e-829990ea22f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.255233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015bbd37-0454-4747-864e-829990ea22f7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.273169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 790/1578] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:21 2022] POST /identity/v3/projects => generated 377 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:21.274253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568cd05-4f94-4984-9fb3-299b00b156ff admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.283790 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 789/1579] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:21 2022] POST /identity/v3/users => generated 294 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:21.284856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bbd37-0454-4747-864e-829990ea22f7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.355518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bbd37-0454-4747-864e-829990ea22f7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.378564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568cd05-4f94-4984-9fb3-299b00b156ff admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.382398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:21.383943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599739e-aa37-45b7-a7fb-17c13acb36d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.384739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599739e-aa37-45b7-a7fb-17c13acb36d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.384977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599739e-aa37-45b7-a7fb-17c13acb36d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.385388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599739e-aa37-45b7-a7fb-17c13acb36d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.386150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599739e-aa37-45b7-a7fb-17c13acb36d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.388350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599739e-aa37-45b7-a7fb-17c13acb36d4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.398726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:21.414565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 791/1580] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:21 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:21.418812 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599739e-aa37-45b7-a7fb-17c13acb36d4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.430379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599739e-aa37-45b7-a7fb-17c13acb36d4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.436385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428911b-ff44-4870-9363-c3781fc8607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.437088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428911b-ff44-4870-9363-c3781fc8607b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.437372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428911b-ff44-4870-9363-c3781fc8607b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.437693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428911b-ff44-4870-9363-c3781fc8607b admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.438422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428911b-ff44-4870-9363-c3781fc8607b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.440503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428911b-ff44-4870-9363-c3781fc8607b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.466170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51313d0-b2f3-4386-b50b-f5991144f572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.466927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51313d0-b2f3-4386-b50b-f5991144f572 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.467132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51313d0-b2f3-4386-b50b-f5991144f572 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.467400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51313d0-b2f3-4386-b50b-f5991144f572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.471070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 792/1581] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:21 2022] POST /identity/v3/users => generated 293 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:21.472126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428911b-ff44-4870-9363-c3781fc8607b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.486730 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51313d0-b2f3-4386-b50b-f5991144f572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.488416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51313d0-b2f3-4386-b50b-f5991144f572 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.490063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428911b-ff44-4870-9363-c3781fc8607b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.496150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ec455-793b-4d89-8ddc-44c5dd38a3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.497284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ec455-793b-4d89-8ddc-44c5dd38a3ae admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.497815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ec455-793b-4d89-8ddc-44c5dd38a3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.498291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ec455-793b-4d89-8ddc-44c5dd38a3ae admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.499111 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ec455-793b-4d89-8ddc-44c5dd38a3ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.501383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ec455-793b-4d89-8ddc-44c5dd38a3ae admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.504428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 790/1582] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:21.511618 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51313d0-b2f3-4386-b50b-f5991144f572 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.528355 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51313d0-b2f3-4386-b50b-f5991144f572 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.531038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 793/1583] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:21 2022] POST /identity/v3/users => generated 293 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:21.531853 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ec455-793b-4d89-8ddc-44c5dd38a3ae admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.538370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a6068-370c-48de-9b2f-d78b9f8b29cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.539955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a6068-370c-48de-9b2f-d78b9f8b29cf admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.540159 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a6068-370c-48de-9b2f-d78b9f8b29cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.540792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a6068-370c-48de-9b2f-d78b9f8b29cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.542013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a6068-370c-48de-9b2f-d78b9f8b29cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.544576 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ec455-793b-4d89-8ddc-44c5dd38a3ae admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.545001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a6068-370c-48de-9b2f-d78b9f8b29cf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.550894 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f05ca-2336-4dae-93b6-49106ff23306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.551810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f05ca-2336-4dae-93b6-49106ff23306 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.552199 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f05ca-2336-4dae-93b6-49106ff23306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.552541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f05ca-2336-4dae-93b6-49106ff23306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.553398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f05ca-2336-4dae-93b6-49106ff23306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.555418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f05ca-2336-4dae-93b6-49106ff23306 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.562905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 791/1584] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.563710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a6068-370c-48de-9b2f-d78b9f8b29cf admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.573899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 794/1585] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.573899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f05ca-2336-4dae-93b6-49106ff23306 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.580626 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a6068-370c-48de-9b2f-d78b9f8b29cf admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.585648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f05ca-2336-4dae-93b6-49106ff23306 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.588441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969590b2-2458-4213-92c1-903d6b9fe27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.589493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969590b2-2458-4213-92c1-903d6b9fe27a admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.590017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969590b2-2458-4213-92c1-903d6b9fe27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.590574 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969590b2-2458-4213-92c1-903d6b9fe27a admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.591279 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc369a-f186-44f2-b44c-8bcd8fd512b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.591549 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969590b2-2458-4213-92c1-903d6b9fe27a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.592054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc369a-f186-44f2-b44c-8bcd8fd512b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.592412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc369a-f186-44f2-b44c-8bcd8fd512b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.592759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc369a-f186-44f2-b44c-8bcd8fd512b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.593705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc369a-f186-44f2-b44c-8bcd8fd512b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.594302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969590b2-2458-4213-92c1-903d6b9fe27a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.595701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc369a-f186-44f2-b44c-8bcd8fd512b7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.614283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 795/1586] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.616274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc369a-f186-44f2-b44c-8bcd8fd512b7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.629259 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 792/1587] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:21 2022] POST /identity/v3/users => generated 289 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:21.630698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969590b2-2458-4213-92c1-903d6b9fe27a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.637007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc369a-f186-44f2-b44c-8bcd8fd512b7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.642777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969590b2-2458-4213-92c1-903d6b9fe27a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.644725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac64abc-b19d-4e18-b85b-e344c0d09877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.645757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac64abc-b19d-4e18-b85b-e344c0d09877 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.646260 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac64abc-b19d-4e18-b85b-e344c0d09877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.646819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac64abc-b19d-4e18-b85b-e344c0d09877 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.648038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac64abc-b19d-4e18-b85b-e344c0d09877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.650031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca011f-33de-48d7-8807-6ea532a545d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.650828 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca011f-33de-48d7-8807-6ea532a545d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.651002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac64abc-b19d-4e18-b85b-e344c0d09877 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.651213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca011f-33de-48d7-8807-6ea532a545d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.651548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca011f-33de-48d7-8807-6ea532a545d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.652386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca011f-33de-48d7-8807-6ea532a545d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.654801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca011f-33de-48d7-8807-6ea532a545d3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.672603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 796/1588] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.672603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac64abc-b19d-4e18-b85b-e344c0d09877 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.673035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 793/1589] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.673035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca011f-33de-48d7-8807-6ea532a545d3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.685813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca011f-33de-48d7-8807-6ea532a545d3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.688941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac64abc-b19d-4e18-b85b-e344c0d09877 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.692133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2aec6-5c3a-40ee-93b3-1f772d273831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.693124 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2aec6-5c3a-40ee-93b3-1f772d273831 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.693595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2aec6-5c3a-40ee-93b3-1f772d273831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.693988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2aec6-5c3a-40ee-93b3-1f772d273831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.695108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2aec6-5c3a-40ee-93b3-1f772d273831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.697375 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2aec6-5c3a-40ee-93b3-1f772d273831 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.698660 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c6cdf-7ddb-48d0-a1da-9e88216662e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.699852 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c6cdf-7ddb-48d0-a1da-9e88216662e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.700518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c6cdf-7ddb-48d0-a1da-9e88216662e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.700935 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c6cdf-7ddb-48d0-a1da-9e88216662e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.702567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c6cdf-7ddb-48d0-a1da-9e88216662e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.705042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c6cdf-7ddb-48d0-a1da-9e88216662e4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.713067 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 794/1590] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.715954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2aec6-5c3a-40ee-93b3-1f772d273831 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.725393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 797/1591] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.726798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c6cdf-7ddb-48d0-a1da-9e88216662e4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.736310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2aec6-5c3a-40ee-93b3-1f772d273831 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.745511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0b29c-1d26-4def-8b6e-8c35717fb6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.746733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0b29c-1d26-4def-8b6e-8c35717fb6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.746998 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c6cdf-7ddb-48d0-a1da-9e88216662e4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.747851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0b29c-1d26-4def-8b6e-8c35717fb6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.748148 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0b29c-1d26-4def-8b6e-8c35717fb6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.749483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0b29c-1d26-4def-8b6e-8c35717fb6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.753451 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0b29c-1d26-4def-8b6e-8c35717fb6b7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.755570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4e1cf-e825-4998-8965-e9651a53a837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.756523 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4e1cf-e825-4998-8965-e9651a53a837 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.757174 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4e1cf-e825-4998-8965-e9651a53a837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.757479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4e1cf-e825-4998-8965-e9651a53a837 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.758734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4e1cf-e825-4998-8965-e9651a53a837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.763041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4e1cf-e825-4998-8965-e9651a53a837 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.776520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 795/1592] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.778205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0b29c-1d26-4def-8b6e-8c35717fb6b7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.788409 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 798/1593] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.790036 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0b29c-1d26-4def-8b6e-8c35717fb6b7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.790298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4e1cf-e825-4998-8965-e9651a53a837 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.795889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a76860-7586-450a-8bb0-44e0d24334f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.796635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a76860-7586-450a-8bb0-44e0d24334f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.796921 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a76860-7586-450a-8bb0-44e0d24334f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.797224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a76860-7586-450a-8bb0-44e0d24334f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.798047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a76860-7586-450a-8bb0-44e0d24334f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.806539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4e1cf-e825-4998-8965-e9651a53a837 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.809217 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a76860-7586-450a-8bb0-44e0d24334f1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.812880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c96b6a-932e-4460-8546-2e8d8c30a15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.814151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c96b6a-932e-4460-8546-2e8d8c30a15b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.814560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c96b6a-932e-4460-8546-2e8d8c30a15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.814993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c96b6a-932e-4460-8546-2e8d8c30a15b admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.816460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c96b6a-932e-4460-8546-2e8d8c30a15b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.819421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c96b6a-932e-4460-8546-2e8d8c30a15b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.826737 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 796/1594] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.828009 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a76860-7586-450a-8bb0-44e0d24334f1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.846055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a76860-7586-450a-8bb0-44e0d24334f1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.852573 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ee1dc-4660-46d7-9e53-47424b5dd09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.853763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ee1dc-4660-46d7-9e53-47424b5dd09f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.854200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ee1dc-4660-46d7-9e53-47424b5dd09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.854642 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ee1dc-4660-46d7-9e53-47424b5dd09f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.855566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ee1dc-4660-46d7-9e53-47424b5dd09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.857399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ee1dc-4660-46d7-9e53-47424b5dd09f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.863867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 799/1595] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.867091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c96b6a-932e-4460-8546-2e8d8c30a15b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.883424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 797/1596] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.883424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ee1dc-4660-46d7-9e53-47424b5dd09f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.889474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c96b6a-932e-4460-8546-2e8d8c30a15b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.898955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51df457-4ecc-47f2-be29-216c0e96fc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.900918 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51df457-4ecc-47f2-be29-216c0e96fc2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.901400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51df457-4ecc-47f2-be29-216c0e96fc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.902066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51df457-4ecc-47f2-be29-216c0e96fc2b admin admin] PATH_INFO: `/v3/projects/ce6e1607faa34ea7ad6a3b6322c4952c/users/9a51d6463e894d219702768e64c612d5/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.902444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ee1dc-4660-46d7-9e53-47424b5dd09f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.915334 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f485d-a3dc-4f99-b3dd-4f90ce182e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:21.916423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f485d-a3dc-4f99-b3dd-4f90ce182e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:21.917133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f485d-a3dc-4f99-b3dd-4f90ce182e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:21.917655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f485d-a3dc-4f99-b3dd-4f90ce182e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:21.918995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f485d-a3dc-4f99-b3dd-4f90ce182e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.922700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f485d-a3dc-4f99-b3dd-4f90ce182e2f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.938419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51df457-4ecc-47f2-be29-216c0e96fc2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce6e1607faa34ea7ad6a3b6322c4952c, user_id=9a51d6463e894d219702768e64c612d5, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:21.941372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-d51df457-4ecc-47f2-be29-216c0e96fc2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 08:03:21.941372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 04 08:03:21.941372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: Jul 04 08:03:21.945972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 798/1597] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:21.945972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f485d-a3dc-4f99-b3dd-4f90ce182e2f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:21.946819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51df457-4ecc-47f2-be29-216c0e96fc2b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:21.983348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f485d-a3dc-4f99-b3dd-4f90ce182e2f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:21.984683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 800/1598] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:21 2022] PUT /identity/v3/projects/ce6e1607faa34ea7ad6a3b6322c4952c/users/9a51d6463e894d219702768e64c612d5/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:21.997026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51df457-4ecc-47f2-be29-216c0e96fc2b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:22.092192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51df457-4ecc-47f2-be29-216c0e96fc2b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:22.097470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3135abb-4745-4abe-8d99-10b8c9ab4f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:22.098438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3135abb-4745-4abe-8d99-10b8c9ab4f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:22.099119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3135abb-4745-4abe-8d99-10b8c9ab4f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:22.099404 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3135abb-4745-4abe-8d99-10b8c9ab4f4c admin admin] PATH_INFO: `/v3/projects/e6c10d4a0e6749c0a8482c13afefb241/users/c9f73f73725b43c2bd084fd1ea3e2ab2/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:22.139151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dfc31-a1ab-48b0-9ddf-8a665f3a1543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:22.141302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dfc31-a1ab-48b0-9ddf-8a665f3a1543 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:22.141529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3135abb-4745-4abe-8d99-10b8c9ab4f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c10d4a0e6749c0a8482c13afefb241, user_id=c9f73f73725b43c2bd084fd1ea3e2ab2, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:22.142421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dfc31-a1ab-48b0-9ddf-8a665f3a1543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:22.143486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dfc31-a1ab-48b0-9ddf-8a665f3a1543 admin admin] PATH_INFO: `/v3/projects/7de2b98bf8c340d0b7646468c3594a47/users/31fe6ac873124636977d593ad5dc2b65/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:22.150337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3135abb-4745-4abe-8d99-10b8c9ab4f4c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:22.187167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9dfc31-a1ab-48b0-9ddf-8a665f3a1543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7de2b98bf8c340d0b7646468c3594a47, user_id=31fe6ac873124636977d593ad5dc2b65, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:22.190256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 801/1599] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:21 2022] PUT /identity/v3/projects/e6c10d4a0e6749c0a8482c13afefb241/users/c9f73f73725b43c2bd084fd1ea3e2ab2/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:22.193264 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9dfc31-a1ab-48b0-9ddf-8a665f3a1543 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:22.193769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3135abb-4745-4abe-8d99-10b8c9ab4f4c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:22.219915 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=9,keystone:SELECT=91 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:22.238483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 799/1600] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:21 2022] PUT /identity/v3/projects/7de2b98bf8c340d0b7646468c3594a47/users/31fe6ac873124636977d593ad5dc2b65/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:22.238483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dfc31-a1ab-48b0-9ddf-8a665f3a1543 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:22.261908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:22.359989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dfc31-a1ab-48b0-9ddf-8a665f3a1543 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:22.367445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e360df-8130-494f-93c5-22187747709e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:22.369519 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e360df-8130-494f-93c5-22187747709e admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:22.369900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e360df-8130-494f-93c5-22187747709e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:22.370133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e360df-8130-494f-93c5-22187747709e admin admin] PATH_INFO: `/v3/projects/de5226d001fb47b399172b6633fec844/users/126ba607541e4c4895b47b4b7602ce01/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:22.373587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3135abb-4745-4abe-8d99-10b8c9ab4f4c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:22.382831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdb9ae-1065-4e6a-a480-504207981d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:22.384037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdb9ae-1065-4e6a-a480-504207981d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:22.384507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdb9ae-1065-4e6a-a480-504207981d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:22.385017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdb9ae-1065-4e6a-a480-504207981d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:22.386533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fdb9ae-1065-4e6a-a480-504207981d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:22.389924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fdb9ae-1065-4e6a-a480-504207981d3d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:22.393910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e360df-8130-494f-93c5-22187747709e admin admin] RBAC: Authorizing `identity:create_grant(project_id=de5226d001fb47b399172b6633fec844, user_id=126ba607541e4c4895b47b4b7602ce01, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:22.397648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e360df-8130-494f-93c5-22187747709e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:22.413182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 802/1601] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:22.414752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdb9ae-1065-4e6a-a480-504207981d3d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:22.425670 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 800/1602] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:22 2022] PUT /identity/v3/projects/de5226d001fb47b399172b6633fec844/users/126ba607541e4c4895b47b4b7602ce01/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:22.425670 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e360df-8130-494f-93c5-22187747709e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:22.434855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdb9ae-1065-4e6a-a480-504207981d3d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:22.548548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7168fc-a318-46ac-9ea3-f2ceae3fea47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:22.550361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7168fc-a318-46ac-9ea3-f2ceae3fea47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:22.551135 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7168fc-a318-46ac-9ea3-f2ceae3fea47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:22.551805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7168fc-a318-46ac-9ea3-f2ceae3fea47 admin admin] PATH_INFO: `/v3/projects/4fadf5b15f7e43b4bd2372f22b293518/users/1501eb69e0a447ce83baa5bdee770d1f/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:22.586206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7168fc-a318-46ac-9ea3-f2ceae3fea47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fadf5b15f7e43b4bd2372f22b293518, user_id=1501eb69e0a447ce83baa5bdee770d1f, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:22.589217 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e360df-8130-494f-93c5-22187747709e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:22.591980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7168fc-a318-46ac-9ea3-f2ceae3fea47 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:22.598445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827667b6-15a4-4db0-a0cf-abde1667d520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:22.600634 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827667b6-15a4-4db0-a0cf-abde1667d520 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:22.601560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827667b6-15a4-4db0-a0cf-abde1667d520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:22.602179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827667b6-15a4-4db0-a0cf-abde1667d520 admin admin] PATH_INFO: `/v3/projects/784a07c025314f0ebd6dc72bd77cc5d9/users/7ed5cb5412ba4988a12fefb7aaa824c0/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:22.626074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827667b6-15a4-4db0-a0cf-abde1667d520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=784a07c025314f0ebd6dc72bd77cc5d9, user_id=7ed5cb5412ba4988a12fefb7aaa824c0, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:22.627848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 803/1603] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:22 2022] PUT /identity/v3/projects/4fadf5b15f7e43b4bd2372f22b293518/users/1501eb69e0a447ce83baa5bdee770d1f/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:22.630544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827667b6-15a4-4db0-a0cf-abde1667d520 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:22.635853 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf9367-2ad7-4e90-bdf3-e42200ff4a33 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:22.638430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf9367-2ad7-4e90-bdf3-e42200ff4a33 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:22.638747 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf9367-2ad7-4e90-bdf3-e42200ff4a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:22.658057 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 801/1604] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:22 2022] PUT /identity/v3/projects/784a07c025314f0ebd6dc72bd77cc5d9/users/7ed5cb5412ba4988a12fefb7aaa824c0/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:22.661804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88d9af-807a-4c16-9c4c-e48052c5b589 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:22.662821 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88d9af-807a-4c16-9c4c-e48052c5b589 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:22.663513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88d9af-807a-4c16-9c4c-e48052c5b589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:22.688963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-82cf9367-2ad7-4e90-bdf3-e42200ff4a33 None None] MFA Rules not processed for user `9a51d6463e894d219702768e64c612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:22.690590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-82cf9367-2ad7-4e90-bdf3-e42200ff4a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:22.725179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5f88d9af-807a-4c16-9c4c-e48052c5b589 None None] MFA Rules not processed for user `c9f73f73725b43c2bd084fd1ea3e2ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:22.727564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5f88d9af-807a-4c16-9c4c-e48052c5b589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:22.844674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 804/1605] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:22.847396 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df59d1-7081-40e0-a8fe-7b9f5034e02d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:22.847765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df59d1-7081-40e0-a8fe-7b9f5034e02d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:22.848204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df59d1-7081-40e0-a8fe-7b9f5034e02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:22.874948 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 802/1606] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:22.876124 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88d9af-807a-4c16-9c4c-e48052c5b589 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:22.882718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:22.886840 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-96df59d1-7081-40e0-a8fe-7b9f5034e02d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:22.888812 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-96df59d1-7081-40e0-a8fe-7b9f5034e02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:22.922070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:23.012172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88d9af-807a-4c16-9c4c-e48052c5b589 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:23.021369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f949a2-3a48-4e38-b17c-1e2b9817a029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:23.023374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f949a2-3a48-4e38-b17c-1e2b9817a029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.023863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f949a2-3a48-4e38-b17c-1e2b9817a029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.024350 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f949a2-3a48-4e38-b17c-1e2b9817a029 admin admin] PATH_INFO: `/v3/projects/f7f3b9d038144a2893afd2281e681b89/users/9ac0a3aad50d4f4480ff361980521b28/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.027544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 805/1607] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:23.030369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9098d5f-d3b7-4730-aad3-ca516fd18624 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.031182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9098d5f-d3b7-4730-aad3-ca516fd18624 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.031741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9098d5f-d3b7-4730-aad3-ca516fd18624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.062438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f949a2-3a48-4e38-b17c-1e2b9817a029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7f3b9d038144a2893afd2281e681b89, user_id=9ac0a3aad50d4f4480ff361980521b28, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:23.068995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f949a2-3a48-4e38-b17c-1e2b9817a029 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:23.069616 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a9098d5f-d3b7-4730-aad3-ca516fd18624 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:23.072397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a9098d5f-d3b7-4730-aad3-ca516fd18624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:23.093765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 806/1608] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:23.097486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89b25c-e2e5-4683-9b06-0dacf7c255d9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.097724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89b25c-e2e5-4683-9b06-0dacf7c255d9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.098342 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89b25c-e2e5-4683-9b06-0dacf7c255d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.113129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 803/1609] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:22 2022] PUT /identity/v3/projects/f7f3b9d038144a2893afd2281e681b89/users/9ac0a3aad50d4f4480ff361980521b28/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:23.113129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8b1bc-2ca6-4cb6-aea0-8410b18f3d90 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.113129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8b1bc-2ca6-4cb6-aea0-8410b18f3d90 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.113129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8b1bc-2ca6-4cb6-aea0-8410b18f3d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.148090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4b89b25c-e2e5-4683-9b06-0dacf7c255d9 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:23.150582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4b89b25c-e2e5-4683-9b06-0dacf7c255d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:23.175508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fba8b1bc-2ca6-4cb6-aea0-8410b18f3d90 None None] MFA Rules not processed for user `7ed5cb5412ba4988a12fefb7aaa824c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:23.178032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fba8b1bc-2ca6-4cb6-aea0-8410b18f3d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:23.301938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 807/1610] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:23.302961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89b25c-e2e5-4683-9b06-0dacf7c255d9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:23.334288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 804/1611] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:23.335120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8b1bc-2ca6-4cb6-aea0-8410b18f3d90 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:23.343230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89b25c-e2e5-4683-9b06-0dacf7c255d9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:23.353850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8b1bc-2ca6-4cb6-aea0-8410b18f3d90 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:23.359268 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384a0ef-925c-40a3-ab4f-f9d55190b173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:23.360164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384a0ef-925c-40a3-ab4f-f9d55190b173 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.360712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384a0ef-925c-40a3-ab4f-f9d55190b173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.361043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384a0ef-925c-40a3-ab4f-f9d55190b173 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.361925 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384a0ef-925c-40a3-ab4f-f9d55190b173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:23.362548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53701c-e69a-47a3-8a92-22de6b4806ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:23.363889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53701c-e69a-47a3-8a92-22de6b4806ee admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.364453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384a0ef-925c-40a3-ab4f-f9d55190b173 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:23.365184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53701c-e69a-47a3-8a92-22de6b4806ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.365759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53701c-e69a-47a3-8a92-22de6b4806ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.367071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53701c-e69a-47a3-8a92-22de6b4806ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:23.372212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53701c-e69a-47a3-8a92-22de6b4806ee admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:23.405462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 805/1612] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:23 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:23.407262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384a0ef-925c-40a3-ab4f-f9d55190b173 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:23.469152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 808/1613] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:23 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:23.483605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340aa1b-7ad6-4764-a62c-61411872047d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.487030 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340aa1b-7ad6-4764-a62c-61411872047d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.488188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340aa1b-7ad6-4764-a62c-61411872047d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.520134 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384a0ef-925c-40a3-ab4f-f9d55190b173 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:23.533348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:23.550750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5340aa1b-7ad6-4764-a62c-61411872047d None None] MFA Rules not processed for user `9ac0a3aad50d4f4480ff361980521b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:23.552178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5340aa1b-7ad6-4764-a62c-61411872047d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:23.645444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32c574-392f-49ff-8946-d4e6fa4a62b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:23.647572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32c574-392f-49ff-8946-d4e6fa4a62b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.648103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32c574-392f-49ff-8946-d4e6fa4a62b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.648599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32c574-392f-49ff-8946-d4e6fa4a62b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.673927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 809/1614] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:23.675250 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340aa1b-7ad6-4764-a62c-61411872047d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:23.689084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340aa1b-7ad6-4764-a62c-61411872047d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:23.695525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ff36a-7259-4f03-8b1e-32902ac2c7aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:23.697320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ff36a-7259-4f03-8b1e-32902ac2c7aa service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.697810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ff36a-7259-4f03-8b1e-32902ac2c7aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.698227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ff36a-7259-4f03-8b1e-32902ac2c7aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.731637 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:23.754494 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32c574-392f-49ff-8946-d4e6fa4a62b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:23.756266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32c574-392f-49ff-8946-d4e6fa4a62b5 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:23.771555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 806/1615] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:23.772524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32c574-392f-49ff-8946-d4e6fa4a62b5 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:23.785503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32c574-392f-49ff-8946-d4e6fa4a62b5 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:23.787897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ff36a-7259-4f03-8b1e-32902ac2c7aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:23.789800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ff36a-7259-4f03-8b1e-32902ac2c7aa service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:23.790831 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77145c-bfd9-4838-a26d-b42e8b3799c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:23.792448 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77145c-bfd9-4838-a26d-b42e8b3799c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.792739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77145c-bfd9-4838-a26d-b42e8b3799c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.795084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77145c-bfd9-4838-a26d-b42e8b3799c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.803385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 810/1616] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:23.804768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a60d27-e17b-41de-95f5-67b1b09ba6f5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.805070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a60d27-e17b-41de-95f5-67b1b09ba6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.805346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a60d27-e17b-41de-95f5-67b1b09ba6f5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.807455 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 811/1617] 10.222.0.43 () {54 vars in 961 bytes} [Mon Jul 4 08:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:03:23.808960 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a60d27-e17b-41de-95f5-67b1b09ba6f5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:23.811171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77145c-bfd9-4838-a26d-b42e8b3799c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:23.813022 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77145c-bfd9-4838-a26d-b42e8b3799c7 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:23.824630 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a60d27-e17b-41de-95f5-67b1b09ba6f5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:23.832378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cfc8f-b87f-49d8-a06a-eb849e701d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:23.833133 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cfc8f-b87f-49d8-a06a-eb849e701d06 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.834277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cfc8f-b87f-49d8-a06a-eb849e701d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.834277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cfc8f-b87f-49d8-a06a-eb849e701d06 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.834864 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cfc8f-b87f-49d8-a06a-eb849e701d06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:23.838477 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cfc8f-b87f-49d8-a06a-eb849e701d06 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:23.842562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 807/1618] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:23.843626 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77145c-bfd9-4838-a26d-b42e8b3799c7 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:23.862113 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77145c-bfd9-4838-a26d-b42e8b3799c7 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:23.869903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 812/1619] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:23 2022] POST /identity/v3/users => generated 298 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:23.872104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cfc8f-b87f-49d8-a06a-eb849e701d06 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:23.875322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34619b39-e4e1-4a2e-8df9-77c40d4d5683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:23.876703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34619b39-e4e1-4a2e-8df9-77c40d4d5683 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.877163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34619b39-e4e1-4a2e-8df9-77c40d4d5683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.878399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34619b39-e4e1-4a2e-8df9-77c40d4d5683 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.879360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34619b39-e4e1-4a2e-8df9-77c40d4d5683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:23.883815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34619b39-e4e1-4a2e-8df9-77c40d4d5683 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:23.974091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cfc8f-b87f-49d8-a06a-eb849e701d06 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:23.974091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933831a-4046-413e-9d48-58a1cb6212f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:23.974091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933831a-4046-413e-9d48-58a1cb6212f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.974091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933831a-4046-413e-9d48-58a1cb6212f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.977397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933831a-4046-413e-9d48-58a1cb6212f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.977397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933831a-4046-413e-9d48-58a1cb6212f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:23.978141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 808/1620] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:23 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:23.978141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821c615-aef2-46a7-8725-4a99639893e4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:23.978141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821c615-aef2-46a7-8725-4a99639893e4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:23.978141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821c615-aef2-46a7-8725-4a99639893e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:23.978637 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933831a-4046-413e-9d48-58a1cb6212f2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:24.039690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 813/1621] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:23 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:24.043008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933831a-4046-413e-9d48-58a1cb6212f2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:24.044418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4821c615-aef2-46a7-8725-4a99639893e4 None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:24.046335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4821c615-aef2-46a7-8725-4a99639893e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:24.188520 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933831a-4046-413e-9d48-58a1cb6212f2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:24.196880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5247cf7-ff0b-469f-946a-06cdc984b107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:24.197770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5247cf7-ff0b-469f-946a-06cdc984b107 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:24.198063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5247cf7-ff0b-469f-946a-06cdc984b107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:24.198456 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5247cf7-ff0b-469f-946a-06cdc984b107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:24.199233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5247cf7-ff0b-469f-946a-06cdc984b107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:24.201303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5247cf7-ff0b-469f-946a-06cdc984b107 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:24.216229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 814/1622] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:24.217167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5247cf7-ff0b-469f-946a-06cdc984b107 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:24.239632 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5247cf7-ff0b-469f-946a-06cdc984b107 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:24.245684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 809/1623] 10.222.0.43 () {58 vars in 1058 bytes} [Mon Jul 4 08:03:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:24.245905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036beae-3190-4933-9b9c-d267e689ac84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:24.246632 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036beae-3190-4933-9b9c-d267e689ac84 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:24.246921 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036beae-3190-4933-9b9c-d267e689ac84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:24.247203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036beae-3190-4933-9b9c-d267e689ac84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:24.247450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821c615-aef2-46a7-8725-4a99639893e4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:24.247973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036beae-3190-4933-9b9c-d267e689ac84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:24.249875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036beae-3190-4933-9b9c-d267e689ac84 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:24.261938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821c615-aef2-46a7-8725-4a99639893e4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:24.270298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 815/1624] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:24.270298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036beae-3190-4933-9b9c-d267e689ac84 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:24.270766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fe628-b9b9-4959-9fc1-621ce2c4f929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:24.271151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fe628-b9b9-4959-9fc1-621ce2c4f929 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:24.271540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fe628-b9b9-4959-9fc1-621ce2c4f929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:24.272003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fe628-b9b9-4959-9fc1-621ce2c4f929 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:24.273012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fe628-b9b9-4959-9fc1-621ce2c4f929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:24.276723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fe628-b9b9-4959-9fc1-621ce2c4f929 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:24.297603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036beae-3190-4933-9b9c-d267e689ac84 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:24.310383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04092de-f4ef-478f-b469-475edf32f721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:24.312138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04092de-f4ef-478f-b469-475edf32f721 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:24.313035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04092de-f4ef-478f-b469-475edf32f721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:24.313637 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04092de-f4ef-478f-b469-475edf32f721 admin admin] PATH_INFO: `/v3/projects/a0a9b6584b554925b441e6d6cba74447/users/ac7ae0e24b314823a53af45753be07bb/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:24.319785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 810/1625] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:24 2022] POST /identity/v3/users => generated 295 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:24.319785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fe628-b9b9-4959-9fc1-621ce2c4f929 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:24.328666 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:24.347249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fe628-b9b9-4959-9fc1-621ce2c4f929 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:24.350983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04092de-f4ef-478f-b469-475edf32f721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a9b6584b554925b441e6d6cba74447, user_id=ac7ae0e24b314823a53af45753be07bb, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:24.357348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67474d69-4a98-407e-9af9-bb6cf9c27a0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:24.358069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04092de-f4ef-478f-b469-475edf32f721 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:24.358339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67474d69-4a98-407e-9af9-bb6cf9c27a0c service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:24.359009 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67474d69-4a98-407e-9af9-bb6cf9c27a0c service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:24.359395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67474d69-4a98-407e-9af9-bb6cf9c27a0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:24.387951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 816/1626] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:24 2022] PUT /identity/v3/projects/a0a9b6584b554925b441e6d6cba74447/users/ac7ae0e24b314823a53af45753be07bb/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:24.389186 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04092de-f4ef-478f-b469-475edf32f721 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:24.451198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:24.487564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04092de-f4ef-478f-b469-475edf32f721 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:24.495439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9421e59-19a1-4bcc-875a-e438fce8d4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:24.496932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9421e59-19a1-4bcc-875a-e438fce8d4ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:24.497278 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9421e59-19a1-4bcc-875a-e438fce8d4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:24.497503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67474d69-4a98-407e-9af9-bb6cf9c27a0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:24.497857 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9421e59-19a1-4bcc-875a-e438fce8d4ef admin admin] PATH_INFO: `/v3/projects/c4b03b4f792c4af9bf353830afc57c39/users/e283efbb4ca84b1b96676e46098333fb/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:24.501399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67474d69-4a98-407e-9af9-bb6cf9c27a0c service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:24.520450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9421e59-19a1-4bcc-875a-e438fce8d4ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b03b4f792c4af9bf353830afc57c39, user_id=e283efbb4ca84b1b96676e46098333fb, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:24.524268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9421e59-19a1-4bcc-875a-e438fce8d4ef admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:24.532256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 811/1627] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:24.533998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67474d69-4a98-407e-9af9-bb6cf9c27a0c service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:24.572009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 817/1628] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:24 2022] PUT /identity/v3/projects/c4b03b4f792c4af9bf353830afc57c39/users/e283efbb4ca84b1b96676e46098333fb/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:24.575139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9421e59-19a1-4bcc-875a-e438fce8d4ef admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:24.585228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67474d69-4a98-407e-9af9-bb6cf9c27a0c service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:24.727640 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba64ea-5d88-4453-8f1a-bf2152ed1fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:24.728364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba64ea-5d88-4453-8f1a-bf2152ed1fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:24.728699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba64ea-5d88-4453-8f1a-bf2152ed1fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:24.728985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba64ea-5d88-4453-8f1a-bf2152ed1fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:24.731120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba64ea-5d88-4453-8f1a-bf2152ed1fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:24.735286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba64ea-5d88-4453-8f1a-bf2152ed1fa7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:24.761970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 812/1629] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:24.763391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba64ea-5d88-4453-8f1a-bf2152ed1fa7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:24.780550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba64ea-5d88-4453-8f1a-bf2152ed1fa7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:24.789005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3742c73-8dd5-4b5f-90da-600dc4fdf275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:24.790209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3742c73-8dd5-4b5f-90da-600dc4fdf275 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:24.790731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3742c73-8dd5-4b5f-90da-600dc4fdf275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:24.791293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3742c73-8dd5-4b5f-90da-600dc4fdf275 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:24.792652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3742c73-8dd5-4b5f-90da-600dc4fdf275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:24.795457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3742c73-8dd5-4b5f-90da-600dc4fdf275 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:24.810705 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 813/1630] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:24.814322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3742c73-8dd5-4b5f-90da-600dc4fdf275 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:24.823457 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9421e59-19a1-4bcc-875a-e438fce8d4ef admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:24.839635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3742c73-8dd5-4b5f-90da-600dc4fdf275 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:24.843594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db982545-2f08-4ebe-9103-0db89ca1b5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:24.844406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db982545-2f08-4ebe-9103-0db89ca1b5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:24.844759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db982545-2f08-4ebe-9103-0db89ca1b5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:24.845096 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db982545-2f08-4ebe-9103-0db89ca1b5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:24.845963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db982545-2f08-4ebe-9103-0db89ca1b5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:24.848318 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0513c95-68ad-4ba5-b7e9-bf11324489d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:24.850082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0513c95-68ad-4ba5-b7e9-bf11324489d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:24.850500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0513c95-68ad-4ba5-b7e9-bf11324489d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:24.850868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0513c95-68ad-4ba5-b7e9-bf11324489d9 admin admin] PATH_INFO: `/v3/projects/228aad6cbc6241ebb3465c049d297c83/users/a650b556283e467ba3aca5d8b04f9855/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:24.851544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db982545-2f08-4ebe-9103-0db89ca1b5b2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:24.886069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0513c95-68ad-4ba5-b7e9-bf11324489d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=228aad6cbc6241ebb3465c049d297c83, user_id=a650b556283e467ba3aca5d8b04f9855, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:24.892878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0513c95-68ad-4ba5-b7e9-bf11324489d9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:24.902283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 818/1631] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:24.902446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db982545-2f08-4ebe-9103-0db89ca1b5b2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:24.936505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 814/1632] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:24 2022] PUT /identity/v3/projects/228aad6cbc6241ebb3465c049d297c83/users/a650b556283e467ba3aca5d8b04f9855/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:24.937484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0513c95-68ad-4ba5-b7e9-bf11324489d9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:24.947991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db982545-2f08-4ebe-9103-0db89ca1b5b2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:25.144164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0513c95-68ad-4ba5-b7e9-bf11324489d9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:25.147516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d76d68-318a-421c-9fd1-51adac84ae8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:25.148543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d76d68-318a-421c-9fd1-51adac84ae8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:25.148930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d76d68-318a-421c-9fd1-51adac84ae8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:25.149086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d76d68-318a-421c-9fd1-51adac84ae8e admin admin] PATH_INFO: `/v3/projects/c4b03b4f792c4af9bf353830afc57c39/users/e283efbb4ca84b1b96676e46098333fb/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:25.150925 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45480885-5a92-4db6-973a-9a22b2f1d0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:25.152241 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45480885-5a92-4db6-973a-9a22b2f1d0ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:25.152744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45480885-5a92-4db6-973a-9a22b2f1d0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:25.153263 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45480885-5a92-4db6-973a-9a22b2f1d0ab admin admin] PATH_INFO: `/v3/projects/a0a9b6584b554925b441e6d6cba74447/users/ac7ae0e24b314823a53af45753be07bb/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:25.154872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d76d68-318a-421c-9fd1-51adac84ae8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b03b4f792c4af9bf353830afc57c39, user_id=e283efbb4ca84b1b96676e46098333fb, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:25.156602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45480885-5a92-4db6-973a-9a22b2f1d0ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a9b6584b554925b441e6d6cba74447, user_id=ac7ae0e24b314823a53af45753be07bb, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:25.159781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d76d68-318a-421c-9fd1-51adac84ae8e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:25.163217 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45480885-5a92-4db6-973a-9a22b2f1d0ab admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:25.173566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:25.184432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 819/1633] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:24 2022] PUT /identity/v3/projects/c4b03b4f792c4af9bf353830afc57c39/users/e283efbb4ca84b1b96676e46098333fb/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:25.197567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d76d68-318a-421c-9fd1-51adac84ae8e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:25.207593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 815/1634] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:24 2022] PUT /identity/v3/projects/a0a9b6584b554925b441e6d6cba74447/users/ac7ae0e24b314823a53af45753be07bb/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:25.216448 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d3be9-0cc6-434d-8421-da47de9809c1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:25.217125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d3be9-0cc6-434d-8421-da47de9809c1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:25.218314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d3be9-0cc6-434d-8421-da47de9809c1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:25.222730 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 816/1635] 10.222.0.43 () {54 vars in 912 bytes} [Mon Jul 4 08:03:25 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:03:25.228190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23665221-c21d-4275-827e-dcdad7669db6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:25.229037 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23665221-c21d-4275-827e-dcdad7669db6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:25.229964 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23665221-c21d-4275-827e-dcdad7669db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:25.294160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-23665221-c21d-4275-827e-dcdad7669db6 None None] MFA Rules not processed for user `e283efbb4ca84b1b96676e46098333fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:25.300771 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-23665221-c21d-4275-827e-dcdad7669db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:25.316223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:25.337731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d76d68-318a-421c-9fd1-51adac84ae8e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:25.570470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a153c-a3e4-43cd-a462-e41688f7c62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:25.572682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a153c-a3e4-43cd-a462-e41688f7c62b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:25.572682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a153c-a3e4-43cd-a462-e41688f7c62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:25.574378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a153c-a3e4-43cd-a462-e41688f7c62b admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:25.577697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a153c-a3e4-43cd-a462-e41688f7c62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:25.589769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a153c-a3e4-43cd-a462-e41688f7c62b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:25.629518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 817/1636] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:25.639298 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d1594-14a7-4a65-b5e4-c5add54aac29 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:25.640122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d1594-14a7-4a65-b5e4-c5add54aac29 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:25.641318 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d1594-14a7-4a65-b5e4-c5add54aac29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:25.662016 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 820/1637] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:25.668543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea28dd-fd5b-4289-bc36-b8cc6cb85cf7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:25.669114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea28dd-fd5b-4289-bc36-b8cc6cb85cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:25.676350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea28dd-fd5b-4289-bc36-b8cc6cb85cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:25.748280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4cea28dd-fd5b-4289-bc36-b8cc6cb85cf7 None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:25.751896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0f7d1594-14a7-4a65-b5e4-c5add54aac29 None None] MFA Rules not processed for user `ac7ae0e24b314823a53af45753be07bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:25.756645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0f7d1594-14a7-4a65-b5e4-c5add54aac29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:25.759004 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4cea28dd-fd5b-4289-bc36-b8cc6cb85cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:25.998032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 821/1638] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:03:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:26.000294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea28dd-fd5b-4289-bc36-b8cc6cb85cf7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:26.028707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 818/1639] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:26.028707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d1594-14a7-4a65-b5e4-c5add54aac29 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:26.071978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d1594-14a7-4a65-b5e4-c5add54aac29 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:26.090145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f6377-e7e9-46ae-8d87-d4e9e2d63b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:26.092492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f6377-e7e9-46ae-8d87-d4e9e2d63b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:26.092492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f6377-e7e9-46ae-8d87-d4e9e2d63b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:26.092932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f6377-e7e9-46ae-8d87-d4e9e2d63b44 admin admin] PATH_INFO: `/v3/projects/228aad6cbc6241ebb3465c049d297c83/users/a650b556283e467ba3aca5d8b04f9855/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:26.098836 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f6377-e7e9-46ae-8d87-d4e9e2d63b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=228aad6cbc6241ebb3465c049d297c83, user_id=a650b556283e467ba3aca5d8b04f9855, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:26.108047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f6377-e7e9-46ae-8d87-d4e9e2d63b44 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:26.172689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea28dd-fd5b-4289-bc36-b8cc6cb85cf7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:26.183216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 819/1640] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:26 2022] PUT /identity/v3/projects/228aad6cbc6241ebb3465c049d297c83/users/a650b556283e467ba3aca5d8b04f9855/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:26.187442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f6377-e7e9-46ae-8d87-d4e9e2d63b44 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:26.268545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:26.292480 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261ceb6-d9ea-4474-bf3f-401eb53af130 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:26.295095 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261ceb6-d9ea-4474-bf3f-401eb53af130 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:26.296704 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261ceb6-d9ea-4474-bf3f-401eb53af130 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:26.297477 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261ceb6-d9ea-4474-bf3f-401eb53af130 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:26.425527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f6377-e7e9-46ae-8d87-d4e9e2d63b44 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:26.442891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d2e7f2-ce4c-4920-ac1e-f7deebf90cd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:26.446414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2e7f2-ce4c-4920-ac1e-f7deebf90cd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:26.446761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261ceb6-d9ea-4474-bf3f-401eb53af130 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:26.447166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2e7f2-ce4c-4920-ac1e-f7deebf90cd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:26.449608 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261ceb6-d9ea-4474-bf3f-401eb53af130 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:26.450901 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2e7f2-ce4c-4920-ac1e-f7deebf90cd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:26.467382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 822/1641] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:26.470569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c7c2c-c0e3-4d95-837a-fa638f1479bb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:26.470873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c7c2c-c0e3-4d95-837a-fa638f1479bb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:26.471152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c7c2c-c0e3-4d95-837a-fa638f1479bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:26.483625 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:26.513401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7d1c7c2c-c0e3-4d95-837a-fa638f1479bb None None] MFA Rules not processed for user `a650b556283e467ba3aca5d8b04f9855`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:26.515058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7d1c7c2c-c0e3-4d95-837a-fa638f1479bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:26.665544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 823/1642] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:26.667222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c7c2c-c0e3-4d95-837a-fa638f1479bb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:26.683333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c7c2c-c0e3-4d95-837a-fa638f1479bb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:26.693597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f653bc-a2ad-46fa-a1f3-074bf6a3b7c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:26.696353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f653bc-a2ad-46fa-a1f3-074bf6a3b7c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:26.698297 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f653bc-a2ad-46fa-a1f3-074bf6a3b7c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:26.698698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f653bc-a2ad-46fa-a1f3-074bf6a3b7c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:26.702086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d2e7f2-ce4c-4920-ac1e-f7deebf90cd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:26.704998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d2e7f2-ce4c-4920-ac1e-f7deebf90cd0 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:26.725459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 820/1643] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:26.917337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f653bc-a2ad-46fa-a1f3-074bf6a3b7c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:26.919118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f653bc-a2ad-46fa-a1f3-074bf6a3b7c7 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:26.957110 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 824/1644] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:27.202150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d2e7f2-ce4c-4920-ac1e-f7deebf90cd0 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:27.216433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d2e7f2-ce4c-4920-ac1e-f7deebf90cd0 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:27.228678 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e25d05-d754-421b-9265-44e91cfe4137 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:27.229530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e25d05-d754-421b-9265-44e91cfe4137 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:27.230163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e25d05-d754-421b-9265-44e91cfe4137 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:27.230949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e25d05-d754-421b-9265-44e91cfe4137 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:27.457048 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e25d05-d754-421b-9265-44e91cfe4137 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:27.463066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e25d05-d754-421b-9265-44e91cfe4137 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:27.487036 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 821/1645] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:28.017683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f653bc-a2ad-46fa-a1f3-074bf6a3b7c7 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:28.059615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:28.233076 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f653bc-a2ad-46fa-a1f3-074bf6a3b7c7 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:28.249244 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5effeb-31c7-4271-8d27-8f72e2ba5c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:28.252067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5effeb-31c7-4271-8d27-8f72e2ba5c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:28.253101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5effeb-31c7-4271-8d27-8f72e2ba5c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:28.253428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5effeb-31c7-4271-8d27-8f72e2ba5c7a admin admin] PATH_INFO: `/v3/users/c9f73f73725b43c2bd084fd1ea3e2ab2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:28.259589 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5effeb-31c7-4271-8d27-8f72e2ba5c7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f73f73725b43c2bd084fd1ea3e2ab2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:28.260708 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-0e5effeb-31c7-4271-8d27-8f72e2ba5c7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 08:03:28.260708 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 04 08:03:28.260708 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: Jul 04 08:03:28.265990 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5effeb-31c7-4271-8d27-8f72e2ba5c7a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:28.503248 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0e5effeb-31c7-4271-8d27-8f72e2ba5c7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f73f73725b43c2bd084fd1ea3e2ab2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:28.530063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0e5effeb-31c7-4271-8d27-8f72e2ba5c7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f73f73725b43c2bd084fd1ea3e2ab2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:28.563243 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0e5effeb-31c7-4271-8d27-8f72e2ba5c7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f73f73725b43c2bd084fd1ea3e2ab2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:28.566052 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0e5effeb-31c7-4271-8d27-8f72e2ba5c7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f73f73725b43c2bd084fd1ea3e2ab2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:28.667337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0e5effeb-31c7-4271-8d27-8f72e2ba5c7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f73f73725b43c2bd084fd1ea3e2ab2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:28.709658 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 825/1646] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:03:28 2022] DELETE /identity/v3/users/c9f73f73725b43c2bd084fd1ea3e2ab2 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:28.714586 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806a448-5c2c-4494-b732-994e4301c8dd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:28.715347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806a448-5c2c-4494-b732-994e4301c8dd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:28.715347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806a448-5c2c-4494-b732-994e4301c8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:28.794068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5806a448-5c2c-4494-b732-994e4301c8dd None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:28.802562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5806a448-5c2c-4494-b732-994e4301c8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:29.123210 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 822/1647] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:29.164814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fe2c2-b115-4a2d-9429-6d5e6f45740b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:29.165491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fe2c2-b115-4a2d-9429-6d5e6f45740b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:29.165859 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fe2c2-b115-4a2d-9429-6d5e6f45740b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:29.169049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 826/1648] 10.222.0.43 () {54 vars in 961 bytes} [Mon Jul 4 08:03:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:03:29.185599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50773cd2-ba16-4858-a3ad-30c41b73b2c1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:29.186098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50773cd2-ba16-4858-a3ad-30c41b73b2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:29.186283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50773cd2-ba16-4858-a3ad-30c41b73b2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:29.199196 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fe2c2-b115-4a2d-9429-6d5e6f45740b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:29.204178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-da9fe2c2-b115-4a2d-9429-6d5e6f45740b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:29.270988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-50773cd2-ba16-4858-a3ad-30c41b73b2c1 None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:29.273376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-50773cd2-ba16-4858-a3ad-30c41b73b2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:29.381480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:29.424021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fe2c2-b115-4a2d-9429-6d5e6f45740b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:29.434016 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7dbe9-ce67-4263-901d-06b537822d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:29.434844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7dbe9-ce67-4263-901d-06b537822d5f service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:29.435137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7dbe9-ce67-4263-901d-06b537822d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:29.435342 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7dbe9-ce67-4263-901d-06b537822d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:29.460757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7dbe9-ce67-4263-901d-06b537822d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:29.464725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7dbe9-ce67-4263-901d-06b537822d5f service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:29.489580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 823/1649] 10.222.0.43 () {58 vars in 1058 bytes} [Mon Jul 4 08:03:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:03:29.493753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50773cd2-ba16-4858-a3ad-30c41b73b2c1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:29.498856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-50773cd2-ba16-4858-a3ad-30c41b73b2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:29.499460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 827/1650] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:29.503741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc7b9a-de8e-4b6b-a66f-a9f62c1f81b4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:29.504091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc7b9a-de8e-4b6b-a66f-a9f62c1f81b4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:29.505726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc7b9a-de8e-4b6b-a66f-a9f62c1f81b4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:29.510111 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 828/1651] 10.222.0.43 () {54 vars in 912 bytes} [Mon Jul 4 08:03:29 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:03:29.515808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d296ec-73c7-4529-9cdd-74a2a97b1519 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:29.516185 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d296ec-73c7-4529-9cdd-74a2a97b1519 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:29.516478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d296ec-73c7-4529-9cdd-74a2a97b1519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:29.534226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50773cd2-ba16-4858-a3ad-30c41b73b2c1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:29.548125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a301a-7c14-4c3d-9ada-3e17c361d11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:29.549125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a301a-7c14-4c3d-9ada-3e17c361d11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:29.549457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a301a-7c14-4c3d-9ada-3e17c361d11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:29.549804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a301a-7c14-4c3d-9ada-3e17c361d11d admin admin] PATH_INFO: `/v3/users/9a51d6463e894d219702768e64c612d5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:29.553190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a301a-7c14-4c3d-9ada-3e17c361d11d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a51d6463e894d219702768e64c612d5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:29.559651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-19d296ec-73c7-4529-9cdd-74a2a97b1519 None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:29.560554 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a301a-7c14-4c3d-9ada-3e17c361d11d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:29.563492 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-19d296ec-73c7-4529-9cdd-74a2a97b1519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:29.591498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 829/1652] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:03:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 04 08:03:29.608790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d296ec-73c7-4529-9cdd-74a2a97b1519 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:29.610377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-19d296ec-73c7-4529-9cdd-74a2a97b1519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:29.717099 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae2a301a-7c14-4c3d-9ada-3e17c361d11d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a51d6463e894d219702768e64c612d5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:29.719813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae2a301a-7c14-4c3d-9ada-3e17c361d11d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a51d6463e894d219702768e64c612d5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:29.721662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae2a301a-7c14-4c3d-9ada-3e17c361d11d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a51d6463e894d219702768e64c612d5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:29.761237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae2a301a-7c14-4c3d-9ada-3e17c361d11d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a51d6463e894d219702768e64c612d5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:29.776371 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:29.811103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae2a301a-7c14-4c3d-9ada-3e17c361d11d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a51d6463e894d219702768e64c612d5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:29.855734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 824/1653] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:03:29 2022] DELETE /identity/v3/users/9a51d6463e894d219702768e64c612d5 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:29.860422 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cd45a-b7fc-4a35-960c-6c865ce5fda0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:29.861408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cd45a-b7fc-4a35-960c-6c865ce5fda0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:29.861826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cd45a-b7fc-4a35-960c-6c865ce5fda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:29.884360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d296ec-73c7-4529-9cdd-74a2a97b1519 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:29.922564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-850cd45a-b7fc-4a35-960c-6c865ce5fda0 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:29.928715 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-850cd45a-b7fc-4a35-960c-6c865ce5fda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:30.087269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 825/1654] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:30.098649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678461e-5e52-4dce-b09d-5e7a1394e090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:30.099836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678461e-5e52-4dce-b09d-5e7a1394e090 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:30.100353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678461e-5e52-4dce-b09d-5e7a1394e090 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:30.100974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678461e-5e52-4dce-b09d-5e7a1394e090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:30.113213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c678461e-5e52-4dce-b09d-5e7a1394e090 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:30.369865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678461e-5e52-4dce-b09d-5e7a1394e090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:30.375598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678461e-5e52-4dce-b09d-5e7a1394e090 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:30.476304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 830/1655] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 866 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:03:30.553363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cd45a-b7fc-4a35-960c-6c865ce5fda0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:30.572417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-850cd45a-b7fc-4a35-960c-6c865ce5fda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:30.793466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:30.895791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cd45a-b7fc-4a35-960c-6c865ce5fda0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:30.937118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e98ba-f236-4091-8ccc-75df814892fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:30.937689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e98ba-f236-4091-8ccc-75df814892fb service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:30.938169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e98ba-f236-4091-8ccc-75df814892fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:30.938657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e98ba-f236-4091-8ccc-75df814892fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:31.061295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e98ba-f236-4091-8ccc-75df814892fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:31.064460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e98ba-f236-4091-8ccc-75df814892fb service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:31.130266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 826/1656] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:33.714887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678461e-5e52-4dce-b09d-5e7a1394e090 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:33.717215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c678461e-5e52-4dce-b09d-5e7a1394e090 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:33.760729 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678461e-5e52-4dce-b09d-5e7a1394e090 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:33.768045 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:33.770499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:33.772226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:33.772975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] PATH_INFO: `/v3/projects/e6c10d4a0e6749c0a8482c13afefb241` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:33.775689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c10d4a0e6749c0a8482c13afefb241)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:33.787547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:33.918904 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c10d4a0e6749c0a8482c13afefb241', 'request_id': 'req-59331013-f271-4e5f-b355-32f1c82fb04a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:33.932325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c10d4a0e6749c0a8482c13afefb241', 'request_id': 'req-59331013-f271-4e5f-b355-32f1c82fb04a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:33.951106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c10d4a0e6749c0a8482c13afefb241', 'request_id': 'req-59331013-f271-4e5f-b355-32f1c82fb04a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:33.978097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] The token cache is being invalidate because project e6c10d4a0e6749c0a8482c13afefb241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:03:33.978394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:33.983691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:33.985195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 831/1657] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:03:33 2022] DELETE /identity/v3/projects/e6c10d4a0e6749c0a8482c13afefb241 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:33.990428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e98ba-f236-4091-8ccc-75df814892fb service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:34.015815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e98ba-f236-4091-8ccc-75df814892fb service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:34.193442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:34.195606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:34.196103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:34.196567 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] PATH_INFO: `/v3/users/ac7ae0e24b314823a53af45753be07bb` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:34.198438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac7ae0e24b314823a53af45753be07bb)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:34.201029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:34.337938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac7ae0e24b314823a53af45753be07bb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:34.339246 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac7ae0e24b314823a53af45753be07bb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:34.340515 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac7ae0e24b314823a53af45753be07bb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:34.358814 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac7ae0e24b314823a53af45753be07bb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:34.405224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac7ae0e24b314823a53af45753be07bb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:34.458724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 827/1658] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:03:33 2022] DELETE /identity/v3/users/ac7ae0e24b314823a53af45753be07bb => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:34.742847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:34.744652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:34.770889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59331013-f271-4e5f-b355-32f1c82fb04a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:34.782567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:34.955768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:34.957397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:34.958012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:34.958609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] PATH_INFO: `/v3/projects/ce6e1607faa34ea7ad6a3b6322c4952c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:34.961254 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce6e1607faa34ea7ad6a3b6322c4952c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:34.970372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:35.164760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce6e1607faa34ea7ad6a3b6322c4952c', 'request_id': 'req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:35.180565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce6e1607faa34ea7ad6a3b6322c4952c', 'request_id': 'req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:35.204220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce6e1607faa34ea7ad6a3b6322c4952c', 'request_id': 'req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:35.259383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] The token cache is being invalidate because project ce6e1607faa34ea7ad6a3b6322c4952c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:03:35.259947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:35.263146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:35.265136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 832/1659] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:03:34 2022] DELETE /identity/v3/projects/ce6e1607faa34ea7ad6a3b6322c4952c => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:35.271061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:35.273289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:35.299860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf5c47-93ce-452f-b6ca-644c6ccf180a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:35.326981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:35.330580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:35.505232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04570e-d9d1-45e4-90e2-f21ec5e3a52d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:35.509810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470cd41-f3e9-4411-b423-7dac02ef9aa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:35.510547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470cd41-f3e9-4411-b423-7dac02ef9aa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:35.510791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470cd41-f3e9-4411-b423-7dac02ef9aa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:35.511067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470cd41-f3e9-4411-b423-7dac02ef9aa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:35.511665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc90ee9-d89d-4f57-a354-87c9e439e09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:35.512989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc90ee9-d89d-4f57-a354-87c9e439e09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:35.513205 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8470cd41-f3e9-4411-b423-7dac02ef9aa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:35.513595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc90ee9-d89d-4f57-a354-87c9e439e09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:35.514039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc90ee9-d89d-4f57-a354-87c9e439e09b admin admin] PATH_INFO: `/v3/users/e283efbb4ca84b1b96676e46098333fb` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:35.515597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc90ee9-d89d-4f57-a354-87c9e439e09b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e283efbb4ca84b1b96676e46098333fb)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:35.518746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc90ee9-d89d-4f57-a354-87c9e439e09b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:35.522482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:35.614065 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bdc90ee9-d89d-4f57-a354-87c9e439e09b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e283efbb4ca84b1b96676e46098333fb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:35.615789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bdc90ee9-d89d-4f57-a354-87c9e439e09b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e283efbb4ca84b1b96676e46098333fb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:35.616739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bdc90ee9-d89d-4f57-a354-87c9e439e09b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e283efbb4ca84b1b96676e46098333fb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:35.633396 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:35.637608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bdc90ee9-d89d-4f57-a354-87c9e439e09b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e283efbb4ca84b1b96676e46098333fb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:35.666937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470cd41-f3e9-4411-b423-7dac02ef9aa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:35.667838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bdc90ee9-d89d-4f57-a354-87c9e439e09b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e283efbb4ca84b1b96676e46098333fb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:35.669481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470cd41-f3e9-4411-b423-7dac02ef9aa3 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:35.691431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 833/1660] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:35.706077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 828/1661] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:03:35 2022] DELETE /identity/v3/users/e283efbb4ca84b1b96676e46098333fb => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:36.051661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470cd41-f3e9-4411-b423-7dac02ef9aa3 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:36.055661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8470cd41-f3e9-4411-b423-7dac02ef9aa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:36.096329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470cd41-f3e9-4411-b423-7dac02ef9aa3 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:36.309412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61e51c-2032-4822-be9e-4c2046b9c79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:36.311616 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61e51c-2032-4822-be9e-4c2046b9c79f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:36.312150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61e51c-2032-4822-be9e-4c2046b9c79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:36.312442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61e51c-2032-4822-be9e-4c2046b9c79f admin admin] PATH_INFO: `/v3/projects/a0a9b6584b554925b441e6d6cba74447` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:36.317221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61e51c-2032-4822-be9e-4c2046b9c79f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a9b6584b554925b441e6d6cba74447)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:36.323228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61e51c-2032-4822-be9e-4c2046b9c79f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:36.517234 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa61e51c-2032-4822-be9e-4c2046b9c79f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a9b6584b554925b441e6d6cba74447', 'request_id': 'req-aa61e51c-2032-4822-be9e-4c2046b9c79f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:36.538118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa61e51c-2032-4822-be9e-4c2046b9c79f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a9b6584b554925b441e6d6cba74447', 'request_id': 'req-aa61e51c-2032-4822-be9e-4c2046b9c79f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:36.555279 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa61e51c-2032-4822-be9e-4c2046b9c79f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a9b6584b554925b441e6d6cba74447', 'request_id': 'req-aa61e51c-2032-4822-be9e-4c2046b9c79f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:36.580343 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa61e51c-2032-4822-be9e-4c2046b9c79f admin admin] The token cache is being invalidate because project a0a9b6584b554925b441e6d6cba74447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:03:36.580649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa61e51c-2032-4822-be9e-4c2046b9c79f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:36.581543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa61e51c-2032-4822-be9e-4c2046b9c79f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:36.584823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 834/1662] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:03:36 2022] DELETE /identity/v3/projects/a0a9b6584b554925b441e6d6cba74447 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:36.594077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66208550-c9e5-42bd-bb90-9623db98bfb0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:36.594831 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66208550-c9e5-42bd-bb90-9623db98bfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:36.595138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66208550-c9e5-42bd-bb90-9623db98bfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:36.720858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-66208550-c9e5-42bd-bb90-9623db98bfb0 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:36.725525 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-66208550-c9e5-42bd-bb90-9623db98bfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:36.904360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1f9f5-4c1f-4028-a758-9cc8137d2278 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:36.905157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1f9f5-4c1f-4028-a758-9cc8137d2278 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:36.906478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1f9f5-4c1f-4028-a758-9cc8137d2278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:36.910085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 829/1663] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:36.985698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e2e1f9f5-4c1f-4028-a758-9cc8137d2278 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:36.989359 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e2e1f9f5-4c1f-4028-a758-9cc8137d2278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:37.013004 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 835/1664] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:37.017811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66208550-c9e5-42bd-bb90-9623db98bfb0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:37.041978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66208550-c9e5-42bd-bb90-9623db98bfb0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:37.058058 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9ca8f-fa16-4b2d-b75f-d536f41d1cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:37.060498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9ca8f-fa16-4b2d-b75f-d536f41d1cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:37.061807 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9ca8f-fa16-4b2d-b75f-d536f41d1cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:37.062723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9ca8f-fa16-4b2d-b75f-d536f41d1cd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:37.064801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9ca8f-fa16-4b2d-b75f-d536f41d1cd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:03:37.065360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9ca8f-fa16-4b2d-b75f-d536f41d1cd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:37.070883 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9ca8f-fa16-4b2d-b75f-d536f41d1cd6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:37.107223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 830/1665] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:03:37.114084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1f9f5-4c1f-4028-a758-9cc8137d2278 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:37.128735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9ca8f-fa16-4b2d-b75f-d536f41d1cd6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:37.134978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-76a9ca8f-fa16-4b2d-b75f-d536f41d1cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:37.139512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1f9f5-4c1f-4028-a758-9cc8137d2278 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:37.157422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90123e2-586d-4c6a-9201-b27b91391bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:37.158224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90123e2-586d-4c6a-9201-b27b91391bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:37.158492 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90123e2-586d-4c6a-9201-b27b91391bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:37.158776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90123e2-586d-4c6a-9201-b27b91391bf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:37.159508 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90123e2-586d-4c6a-9201-b27b91391bf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:37.162906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90123e2-586d-4c6a-9201-b27b91391bf9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:37.194521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9ca8f-fa16-4b2d-b75f-d536f41d1cd6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:37.210473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:37.211975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:37.212446 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:37.212962 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] PATH_INFO: `/v3/projects/c4b03b4f792c4af9bf353830afc57c39` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:37.216536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b03b4f792c4af9bf353830afc57c39)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:37.219617 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 08:03:37.219617 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Jul 04 08:03:37.219617 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: Jul 04 08:03:37.222014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:37.226305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 836/1666] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:37 2022] POST /identity/v3/projects => generated 393 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:37.227532 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90123e2-586d-4c6a-9201-b27b91391bf9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:37.372373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b03b4f792c4af9bf353830afc57c39', 'request_id': 'req-9a634f98-fb80-4bf7-9b93-e116f4db39ba'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:37.399179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b03b4f792c4af9bf353830afc57c39', 'request_id': 'req-9a634f98-fb80-4bf7-9b93-e116f4db39ba'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:37.407340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:37.444734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:37.472671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b03b4f792c4af9bf353830afc57c39', 'request_id': 'req-9a634f98-fb80-4bf7-9b93-e116f4db39ba'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:37.507555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] The token cache is being invalidate because project c4b03b4f792c4af9bf353830afc57c39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:03:37.508296 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:37.512701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9a634f98-fb80-4bf7-9b93-e116f4db39ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:37.520679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 831/1667] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:03:37 2022] DELETE /identity/v3/projects/c4b03b4f792c4af9bf353830afc57c39 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:37.528212 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d3342-cdcf-47e4-ac14-2b97493a6602 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:37.528829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d3342-cdcf-47e4-ac14-2b97493a6602 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:37.536265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d3342-cdcf-47e4-ac14-2b97493a6602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:37.594367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90123e2-586d-4c6a-9201-b27b91391bf9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:37.626890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-708d3342-cdcf-47e4-ac14-2b97493a6602 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:37.635981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-708d3342-cdcf-47e4-ac14-2b97493a6602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:37.857607 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc488b-a0d6-4461-b54f-8d1747303a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:37.858667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc488b-a0d6-4461-b54f-8d1747303a6d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:37.858914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc488b-a0d6-4461-b54f-8d1747303a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:37.859419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc488b-a0d6-4461-b54f-8d1747303a6d admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:37.861979 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc488b-a0d6-4461-b54f-8d1747303a6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:37.869845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc488b-a0d6-4461-b54f-8d1747303a6d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:37.916163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 832/1668] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:37.931871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f48b9-fe33-49b6-981e-7e5443e01909 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:37.934081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f48b9-fe33-49b6-981e-7e5443e01909 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:37.941917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f48b9-fe33-49b6-981e-7e5443e01909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:37.945869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 837/1669] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:37 2022] POST /identity/v3/users => generated 297 bytes in 719 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:37.948869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc488b-a0d6-4461-b54f-8d1747303a6d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:37.955858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dccc488b-a0d6-4461-b54f-8d1747303a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:37.983948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc488b-a0d6-4461-b54f-8d1747303a6d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:37.991905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f60f1a-dcbe-471c-9d99-397ee0177fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:37.995192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f60f1a-dcbe-471c-9d99-397ee0177fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:37.995552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f60f1a-dcbe-471c-9d99-397ee0177fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:37.997135 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f60f1a-dcbe-471c-9d99-397ee0177fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:37.998273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f60f1a-dcbe-471c-9d99-397ee0177fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:38.005652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f60f1a-dcbe-471c-9d99-397ee0177fb5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:38.043756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b52f48b9-fe33-49b6-981e-7e5443e01909 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:38.045342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b52f48b9-fe33-49b6-981e-7e5443e01909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:38.045953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 838/1670] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:38.047226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f60f1a-dcbe-471c-9d99-397ee0177fb5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:38.079460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 833/1671] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:03:38.080649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f48b9-fe33-49b6-981e-7e5443e01909 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:38.081258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f60f1a-dcbe-471c-9d99-397ee0177fb5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:38.097152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efff91a-3cb0-4fb8-b734-125e44396937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:38.098330 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efff91a-3cb0-4fb8-b734-125e44396937 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:38.098764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efff91a-3cb0-4fb8-b734-125e44396937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:38.099156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efff91a-3cb0-4fb8-b734-125e44396937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:38.100351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efff91a-3cb0-4fb8-b734-125e44396937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:38.102769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efff91a-3cb0-4fb8-b734-125e44396937 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:38.135819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f48b9-fe33-49b6-981e-7e5443e01909 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:38.156384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716cdb4-f6f8-42e3-abc9-8c1f10bf3cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:38.157877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716cdb4-f6f8-42e3-abc9-8c1f10bf3cce admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:38.158186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716cdb4-f6f8-42e3-abc9-8c1f10bf3cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:38.158479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716cdb4-f6f8-42e3-abc9-8c1f10bf3cce admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:38.159288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716cdb4-f6f8-42e3-abc9-8c1f10bf3cce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:03:38.159615 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716cdb4-f6f8-42e3-abc9-8c1f10bf3cce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:38.167091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716cdb4-f6f8-42e3-abc9-8c1f10bf3cce admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:38.173176 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 839/1672] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:38.180255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efff91a-3cb0-4fb8-b734-125e44396937 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:38.249801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efff91a-3cb0-4fb8-b734-125e44396937 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:38.250340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 834/1673] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:03:38.253688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716cdb4-f6f8-42e3-abc9-8c1f10bf3cce admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:38.283742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a0f8c-2161-47a2-b2c1-e170979024c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:38.286440 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a0f8c-2161-47a2-b2c1-e170979024c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:38.287243 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a0f8c-2161-47a2-b2c1-e170979024c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:38.287750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a0f8c-2161-47a2-b2c1-e170979024c7 admin admin] PATH_INFO: `/v3/projects/0232db2998e5444987fe8886b870957e/users/21aa08cb7bbc44928aa0e29085e7d44f/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:38.298095 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716cdb4-f6f8-42e3-abc9-8c1f10bf3cce admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:38.315540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feaf7a7-4768-40f1-89b9-c39dbac779d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:38.321285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feaf7a7-4768-40f1-89b9-c39dbac779d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:38.322175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feaf7a7-4768-40f1-89b9-c39dbac779d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:38.322677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feaf7a7-4768-40f1-89b9-c39dbac779d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:38.324220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feaf7a7-4768-40f1-89b9-c39dbac779d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:38.328783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feaf7a7-4768-40f1-89b9-c39dbac779d9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:38.332064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a0f8c-2161-47a2-b2c1-e170979024c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0232db2998e5444987fe8886b870957e, user_id=21aa08cb7bbc44928aa0e29085e7d44f, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:38.339495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a0f8c-2161-47a2-b2c1-e170979024c7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:38.379170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 840/1674] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:38 2022] PUT /identity/v3/projects/0232db2998e5444987fe8886b870957e/users/21aa08cb7bbc44928aa0e29085e7d44f/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:38.405219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca39ee-4841-4073-a9b9-db65c7931891 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:38.405219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca39ee-4841-4073-a9b9-db65c7931891 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:38.405732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca39ee-4841-4073-a9b9-db65c7931891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:38.416366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 835/1675] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:38 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:38.421098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feaf7a7-4768-40f1-89b9-c39dbac779d9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:38.533324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d2ca39ee-4841-4073-a9b9-db65c7931891 None None] MFA Rules not processed for user `21aa08cb7bbc44928aa0e29085e7d44f`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:38.535439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d2ca39ee-4841-4073-a9b9-db65c7931891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:38.540141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feaf7a7-4768-40f1-89b9-c39dbac779d9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:38.548678 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073e909-8d6b-4fb7-a082-2d6dc040e2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:38.549991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073e909-8d6b-4fb7-a082-2d6dc040e2c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:38.550420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073e909-8d6b-4fb7-a082-2d6dc040e2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:38.550851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073e909-8d6b-4fb7-a082-2d6dc040e2c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:38.553313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073e909-8d6b-4fb7-a082-2d6dc040e2c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:38.555910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073e909-8d6b-4fb7-a082-2d6dc040e2c1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:38.599790 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 836/1676] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:38 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:38.601162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073e909-8d6b-4fb7-a082-2d6dc040e2c1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:38.627884 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073e909-8d6b-4fb7-a082-2d6dc040e2c1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:38.641497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2158d-e451-44ee-b55d-33a905723173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:38.644585 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2158d-e451-44ee-b55d-33a905723173 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:38.645099 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2158d-e451-44ee-b55d-33a905723173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:38.648179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2158d-e451-44ee-b55d-33a905723173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:38.649161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2158d-e451-44ee-b55d-33a905723173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:38.654115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2158d-e451-44ee-b55d-33a905723173 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:38.692835 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 837/1677] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:38.694690 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2158d-e451-44ee-b55d-33a905723173 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:38.719377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2158d-e451-44ee-b55d-33a905723173 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:38.732770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd444c-fbec-4bbd-be74-7c705fdbda14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:38.733603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd444c-fbec-4bbd-be74-7c705fdbda14 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:38.735305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd444c-fbec-4bbd-be74-7c705fdbda14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:38.736092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd444c-fbec-4bbd-be74-7c705fdbda14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:38.737665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd444c-fbec-4bbd-be74-7c705fdbda14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:38.748373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd444c-fbec-4bbd-be74-7c705fdbda14 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:38.748666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 841/1678] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:38.751596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca39ee-4841-4073-a9b9-db65c7931891 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:38.781563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 838/1679] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:38.783088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca39ee-4841-4073-a9b9-db65c7931891 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:38.785128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd444c-fbec-4bbd-be74-7c705fdbda14 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:38.798865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795295f3-8b8d-44ff-b457-435199a21ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:38.800123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795295f3-8b8d-44ff-b457-435199a21ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:38.800457 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795295f3-8b8d-44ff-b457-435199a21ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:38.803246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795295f3-8b8d-44ff-b457-435199a21ab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:38.806243 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795295f3-8b8d-44ff-b457-435199a21ab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:38.809211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd444c-fbec-4bbd-be74-7c705fdbda14 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:38.810849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795295f3-8b8d-44ff-b457-435199a21ab9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:38.822867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcd34b-808c-485c-a73b-80b941d59180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:38.824322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcd34b-808c-485c-a73b-80b941d59180 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:38.827789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcd34b-808c-485c-a73b-80b941d59180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:38.828216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcd34b-808c-485c-a73b-80b941d59180 admin admin] PATH_INFO: `/v3/projects/1d9903110fae40a9932b10cd8aba3a0c/users/cfbaae1c8e8f4a3984bc169da9170515/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:38.900077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:38.916482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bcd34b-808c-485c-a73b-80b941d59180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9903110fae40a9932b10cd8aba3a0c, user_id=cfbaae1c8e8f4a3984bc169da9170515, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:38.932533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 842/1680] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:38 2022] POST /identity/v3/projects => generated 393 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:03:38.934571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795295f3-8b8d-44ff-b457-435199a21ab9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:38.947150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bcd34b-808c-485c-a73b-80b941d59180 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:39.007218 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:39.015602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 839/1681] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:38 2022] PUT /identity/v3/projects/1d9903110fae40a9932b10cd8aba3a0c/users/cfbaae1c8e8f4a3984bc169da9170515/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:39.057083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a255ec7-3292-426d-aace-94870aa9b580 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:39.059871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a255ec7-3292-426d-aace-94870aa9b580 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:39.066112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a255ec7-3292-426d-aace-94870aa9b580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:39.211419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9a255ec7-3292-426d-aace-94870aa9b580 None None] MFA Rules not processed for user `cfbaae1c8e8f4a3984bc169da9170515`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:39.212826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9a255ec7-3292-426d-aace-94870aa9b580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:39.307622 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795295f3-8b8d-44ff-b457-435199a21ab9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:39.477078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a895a-87ef-476b-8813-75cb225c2fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:39.478127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a895a-87ef-476b-8813-75cb225c2fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:39.478417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a895a-87ef-476b-8813-75cb225c2fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:39.478702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a895a-87ef-476b-8813-75cb225c2fc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:39.479438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a895a-87ef-476b-8813-75cb225c2fc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:39.485078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a895a-87ef-476b-8813-75cb225c2fc2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:39.541417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 843/1682] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:38 2022] POST /identity/v3/users => generated 303 bytes in 608 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:39.547364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a895a-87ef-476b-8813-75cb225c2fc2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:39.570680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a895a-87ef-476b-8813-75cb225c2fc2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:39.583239 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456540fe-738f-4d48-87a3-6c1979f7d558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:39.583978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456540fe-738f-4d48-87a3-6c1979f7d558 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:39.584269 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456540fe-738f-4d48-87a3-6c1979f7d558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:39.584560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456540fe-738f-4d48-87a3-6c1979f7d558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:39.585351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456540fe-738f-4d48-87a3-6c1979f7d558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:39.589670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456540fe-738f-4d48-87a3-6c1979f7d558 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:39.616570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 844/1683] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:39.621037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456540fe-738f-4d48-87a3-6c1979f7d558 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:39.638653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456540fe-738f-4d48-87a3-6c1979f7d558 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:39.652886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47db47-2717-48f6-9928-fe3b435a91a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:39.653877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47db47-2717-48f6-9928-fe3b435a91a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:39.654165 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47db47-2717-48f6-9928-fe3b435a91a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:39.654446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47db47-2717-48f6-9928-fe3b435a91a8 admin admin] PATH_INFO: `/v3/projects/ab316665fbb942fe8858d99acb2622f4/users/154ea0e85a124e6da43a676744b5eeb4/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:39.687762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 840/1684] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 638 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:39.697098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a255ec7-3292-426d-aace-94870aa9b580 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:39.702623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47db47-2717-48f6-9928-fe3b435a91a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab316665fbb942fe8858d99acb2622f4, user_id=154ea0e85a124e6da43a676744b5eeb4, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:39.708150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47db47-2717-48f6-9928-fe3b435a91a8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:39.726353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a255ec7-3292-426d-aace-94870aa9b580 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:39.739225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 845/1685] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:39 2022] PUT /identity/v3/projects/ab316665fbb942fe8858d99acb2622f4/users/154ea0e85a124e6da43a676744b5eeb4/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:03:39.744032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32375419-344d-40cc-b214-4ac19194a029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:39.744032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32375419-344d-40cc-b214-4ac19194a029 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:39.744032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32375419-344d-40cc-b214-4ac19194a029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:39.744032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32375419-344d-40cc-b214-4ac19194a029 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:39.745552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47db47-2717-48f6-9928-fe3b435a91a8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:39.746795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32375419-344d-40cc-b214-4ac19194a029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:39.748934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32375419-344d-40cc-b214-4ac19194a029 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:39.833989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 841/1686] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:39 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:03:39.836204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32375419-344d-40cc-b214-4ac19194a029 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:39.952439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47db47-2717-48f6-9928-fe3b435a91a8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:39.985987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:40.031919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32375419-344d-40cc-b214-4ac19194a029 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:40.038702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98200b65-f2fd-440a-be39-b15648c95116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:40.039422 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98200b65-f2fd-440a-be39-b15648c95116 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:40.039701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98200b65-f2fd-440a-be39-b15648c95116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:40.039978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98200b65-f2fd-440a-be39-b15648c95116 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:40.040768 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98200b65-f2fd-440a-be39-b15648c95116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:40.042911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98200b65-f2fd-440a-be39-b15648c95116 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:40.075626 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 842/1687] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:39 2022] POST /identity/v3/users => generated 299 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:40.078110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bba904-007e-40b7-a5c7-e9f87856b868 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:40.078639 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bba904-007e-40b7-a5c7-e9f87856b868 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:40.078923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bba904-007e-40b7-a5c7-e9f87856b868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:40.123540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-34bba904-007e-40b7-a5c7-e9f87856b868 None None] MFA Rules not processed for user `126ba607541e4c4895b47b4b7602ce01`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:40.125164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-34bba904-007e-40b7-a5c7-e9f87856b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:40.211989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d9846-f245-4c9f-a7e6-812ef81d57cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:40.212904 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d9846-f245-4c9f-a7e6-812ef81d57cc admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:40.213414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d9846-f245-4c9f-a7e6-812ef81d57cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:40.215438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d9846-f245-4c9f-a7e6-812ef81d57cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:40.217357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d9846-f245-4c9f-a7e6-812ef81d57cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:40.224878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d9846-f245-4c9f-a7e6-812ef81d57cc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:40.233520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:40.267282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 846/1688] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 522 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:03:40.281255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d9846-f245-4c9f-a7e6-812ef81d57cc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:40.322817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 843/1689] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:40.324514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bba904-007e-40b7-a5c7-e9f87856b868 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:40.356343 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d9846-f245-4c9f-a7e6-812ef81d57cc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:40.392590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883309a-1c0b-429d-8e48-628905250e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:40.393830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883309a-1c0b-429d-8e48-628905250e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:40.394357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883309a-1c0b-429d-8e48-628905250e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:40.394926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883309a-1c0b-429d-8e48-628905250e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:40.395974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bba904-007e-40b7-a5c7-e9f87856b868 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:40.396173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883309a-1c0b-429d-8e48-628905250e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:40.401931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883309a-1c0b-429d-8e48-628905250e4a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:40.416597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68698ecb-1f1f-49dc-b5a8-7b3cd2bad85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:40.416597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68698ecb-1f1f-49dc-b5a8-7b3cd2bad85f admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:40.416597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68698ecb-1f1f-49dc-b5a8-7b3cd2bad85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:40.416597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68698ecb-1f1f-49dc-b5a8-7b3cd2bad85f admin admin] PATH_INFO: `/v3/projects/ab316665fbb942fe8858d99acb2622f4/users/154ea0e85a124e6da43a676744b5eeb4/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:40.420032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68698ecb-1f1f-49dc-b5a8-7b3cd2bad85f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab316665fbb942fe8858d99acb2622f4, user_id=154ea0e85a124e6da43a676744b5eeb4, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:40.425249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68698ecb-1f1f-49dc-b5a8-7b3cd2bad85f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:40.469726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 847/1690] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:40.472377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8871e62-f135-4808-8442-00fc57e194c6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:40.472842 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8871e62-f135-4808-8442-00fc57e194c6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:40.473365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8871e62-f135-4808-8442-00fc57e194c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:40.478973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 844/1691] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:40 2022] PUT /identity/v3/projects/ab316665fbb942fe8858d99acb2622f4/users/154ea0e85a124e6da43a676744b5eeb4/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:40.482556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68698ecb-1f1f-49dc-b5a8-7b3cd2bad85f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:40.486278 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-68698ecb-1f1f-49dc-b5a8-7b3cd2bad85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:40.614667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f8871e62-f135-4808-8442-00fc57e194c6 None None] MFA Rules not processed for user `18b1072539af49729dff7ee9ea445c77`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:40.618952 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f8871e62-f135-4808-8442-00fc57e194c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:40.855337 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68698ecb-1f1f-49dc-b5a8-7b3cd2bad85f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:40.897956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 848/1692] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:40.901238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae9360-9ab2-45be-8349-f105a2f7a105 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:40.902416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae9360-9ab2-45be-8349-f105a2f7a105 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:40.902961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae9360-9ab2-45be-8349-f105a2f7a105 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:40.903513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae9360-9ab2-45be-8349-f105a2f7a105 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:40.904193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8871e62-f135-4808-8442-00fc57e194c6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:41.157607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae9360-9ab2-45be-8349-f105a2f7a105 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:41.166317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae9360-9ab2-45be-8349-f105a2f7a105 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:41.191670 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 845/1693] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:41.199375 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d09dd-f462-460a-ac18-6e0539101409 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:41.208354 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d09dd-f462-460a-ac18-6e0539101409 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:41.208793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d09dd-f462-460a-ac18-6e0539101409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:41.305904 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8871e62-f135-4808-8442-00fc57e194c6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:41.339666 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-dd5d09dd-f462-460a-ac18-6e0539101409 None None] MFA Rules not processed for user `154ea0e85a124e6da43a676744b5eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:41.347465 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89eb5e-6fd6-4571-aff0-2cc5a47f74fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:41.349387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-dd5d09dd-f462-460a-ac18-6e0539101409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:41.351531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89eb5e-6fd6-4571-aff0-2cc5a47f74fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:41.352084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89eb5e-6fd6-4571-aff0-2cc5a47f74fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:41.352627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89eb5e-6fd6-4571-aff0-2cc5a47f74fa admin admin] PATH_INFO: `/v3/projects/b25d24fcf6e84923bfec1ed06ad723db/users/013d0b65d54640fa99ff61bad3ea6173/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:41.489289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89eb5e-6fd6-4571-aff0-2cc5a47f74fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25d24fcf6e84923bfec1ed06ad723db, user_id=013d0b65d54640fa99ff61bad3ea6173, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:41.501042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89eb5e-6fd6-4571-aff0-2cc5a47f74fa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:41.574863 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 849/1694] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:40 2022] PUT /identity/v3/projects/b25d24fcf6e84923bfec1ed06ad723db/users/013d0b65d54640fa99ff61bad3ea6173/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:41.574863 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89eb5e-6fd6-4571-aff0-2cc5a47f74fa admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:41.583650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:41.599511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-af89eb5e-6fd6-4571-aff0-2cc5a47f74fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:41.645183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 846/1695] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:41.655270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d09dd-f462-460a-ac18-6e0539101409 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:41.705027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d09dd-f462-460a-ac18-6e0539101409 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:41.709615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:41.940107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2676eed-fdab-4f46-aa1c-3f754554afa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:41.941185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2676eed-fdab-4f46-aa1c-3f754554afa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:41.941537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2676eed-fdab-4f46-aa1c-3f754554afa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:41.946961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2676eed-fdab-4f46-aa1c-3f754554afa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:41.952852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2676eed-fdab-4f46-aa1c-3f754554afa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:41.957239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2676eed-fdab-4f46-aa1c-3f754554afa8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:42.034690 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 847/1696] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:42.038336 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2676eed-fdab-4f46-aa1c-3f754554afa8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:42.044367 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f2676eed-fdab-4f46-aa1c-3f754554afa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:42.089946 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89eb5e-6fd6-4571-aff0-2cc5a47f74fa admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:42.120802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b752557-c395-4a52-8e65-7015ef62b643 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:42.121507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b752557-c395-4a52-8e65-7015ef62b643 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:42.121889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b752557-c395-4a52-8e65-7015ef62b643 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:42.122227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b752557-c395-4a52-8e65-7015ef62b643 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:42.227080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2676eed-fdab-4f46-aa1c-3f754554afa8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:42.238430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fb0f4-0f01-4e1a-bcfe-519f8907361d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:42.240285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fb0f4-0f01-4e1a-bcfe-519f8907361d service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:42.240704 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fb0f4-0f01-4e1a-bcfe-519f8907361d service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:42.241086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fb0f4-0f01-4e1a-bcfe-519f8907361d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:42.339212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b752557-c395-4a52-8e65-7015ef62b643 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:42.340916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b752557-c395-4a52-8e65-7015ef62b643 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:42.378634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 850/1697] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 807 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:42.379425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b752557-c395-4a52-8e65-7015ef62b643 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:42.420690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b752557-c395-4a52-8e65-7015ef62b643 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:42.431366 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afb97f-e2d8-42ea-b400-a6fadab1791f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:42.432324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afb97f-e2d8-42ea-b400-a6fadab1791f admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:42.432678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afb97f-e2d8-42ea-b400-a6fadab1791f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:42.433000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afb97f-e2d8-42ea-b400-a6fadab1791f admin admin] PATH_INFO: `/v3/projects/b25d24fcf6e84923bfec1ed06ad723db/users/013d0b65d54640fa99ff61bad3ea6173/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:42.446674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afb97f-e2d8-42ea-b400-a6fadab1791f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25d24fcf6e84923bfec1ed06ad723db, user_id=013d0b65d54640fa99ff61bad3ea6173, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:42.449229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afb97f-e2d8-42ea-b400-a6fadab1791f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:42.464842 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fb0f4-0f01-4e1a-bcfe-519f8907361d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:42.485770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 851/1698] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:42 2022] PUT /identity/v3/projects/b25d24fcf6e84923bfec1ed06ad723db/users/013d0b65d54640fa99ff61bad3ea6173/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:42.491841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7660aab-2e9c-45c8-86cb-b70ac9c41c07 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:42.492761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7660aab-2e9c-45c8-86cb-b70ac9c41c07 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:42.492761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7660aab-2e9c-45c8-86cb-b70ac9c41c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:42.493459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fb0f4-0f01-4e1a-bcfe-519f8907361d service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:42.536580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 848/1699] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:42.608816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d7660aab-2e9c-45c8-86cb-b70ac9c41c07 None None] MFA Rules not processed for user `013d0b65d54640fa99ff61bad3ea6173`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:42.620478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d7660aab-2e9c-45c8-86cb-b70ac9c41c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:42.696208 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fb0f4-0f01-4e1a-bcfe-519f8907361d service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:42.731894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fb0f4-0f01-4e1a-bcfe-519f8907361d service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:42.896810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 852/1700] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:42.912796 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:43.026007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5b294-9460-4387-bd6a-d6b7c10f98c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:43.026899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5b294-9460-4387-bd6a-d6b7c10f98c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:43.027190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5b294-9460-4387-bd6a-d6b7c10f98c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:43.027561 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5b294-9460-4387-bd6a-d6b7c10f98c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:43.039091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7660aab-2e9c-45c8-86cb-b70ac9c41c07 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:43.040901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d7660aab-2e9c-45c8-86cb-b70ac9c41c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:43.079335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7660aab-2e9c-45c8-86cb-b70ac9c41c07 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:43.095495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae004824-5ce7-4ed2-a01a-776b1cb305fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:43.096185 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae004824-5ce7-4ed2-a01a-776b1cb305fe service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:43.096466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae004824-5ce7-4ed2-a01a-776b1cb305fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:43.096743 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae004824-5ce7-4ed2-a01a-776b1cb305fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:43.186061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:43.335941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5b294-9460-4387-bd6a-d6b7c10f98c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:43.341649 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5b294-9460-4387-bd6a-d6b7c10f98c7 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:43.374970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae004824-5ce7-4ed2-a01a-776b1cb305fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:43.376504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae004824-5ce7-4ed2-a01a-776b1cb305fe service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:43.387070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 849/1701] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:43.406753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 853/1702] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:03:45.085791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f76f1-faa6-4408-87eb-46d16866c433 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:45.086503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f76f1-faa6-4408-87eb-46d16866c433 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:45.086503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f76f1-faa6-4408-87eb-46d16866c433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:45.196092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2f1f76f1-faa6-4408-87eb-46d16866c433 None None] MFA Rules not processed for user `31fe6ac873124636977d593ad5dc2b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:45.198332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2f1f76f1-faa6-4408-87eb-46d16866c433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:45.641144 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae004824-5ce7-4ed2-a01a-776b1cb305fe service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:45.641617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ae004824-5ce7-4ed2-a01a-776b1cb305fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:45.701462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 850/1703] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 617 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:03:45.727324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1c996-03e7-4914-8bb2-7457c8a39f44 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:45.728626 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1c996-03e7-4914-8bb2-7457c8a39f44 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:45.728923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1c996-03e7-4914-8bb2-7457c8a39f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:45.781457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0ed1c996-03e7-4914-8bb2-7457c8a39f44 None None] MFA Rules not processed for user `31fe6ac873124636977d593ad5dc2b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:45.783683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0ed1c996-03e7-4914-8bb2-7457c8a39f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:45.814467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 851/1704] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:45.831027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1c996-03e7-4914-8bb2-7457c8a39f44 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:45.986046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae004824-5ce7-4ed2-a01a-776b1cb305fe service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:45.999396 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:46.020849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:46.021599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:46.025657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] PATH_INFO: `/v3/users/7ed5cb5412ba4988a12fefb7aaa824c0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:46.030495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed5cb5412ba4988a12fefb7aaa824c0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:46.046390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:46.059792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1c996-03e7-4914-8bb2-7457c8a39f44 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:46.076484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e3f14-038e-4b99-ab33-3ae2f3614be8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:46.077273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e3f14-038e-4b99-ab33-3ae2f3614be8 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:46.077644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e3f14-038e-4b99-ab33-3ae2f3614be8 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:46.078046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e3f14-038e-4b99-ab33-3ae2f3614be8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:46.095149 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:46.106794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e3f14-038e-4b99-ab33-3ae2f3614be8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:46.115570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e3f14-038e-4b99-ab33-3ae2f3614be8 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:46.163456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 852/1705] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:46.217175 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed5cb5412ba4988a12fefb7aaa824c0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:46.240187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed5cb5412ba4988a12fefb7aaa824c0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:46.301604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed5cb5412ba4988a12fefb7aaa824c0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:46.303401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed5cb5412ba4988a12fefb7aaa824c0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:46.382495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed5cb5412ba4988a12fefb7aaa824c0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:46.400131 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 854/1706] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:03:45 2022] DELETE /identity/v3/users/7ed5cb5412ba4988a12fefb7aaa824c0 => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:46.405160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bff85-8abc-4a68-a5c1-6d7e14108d95 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:46.405454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bff85-8abc-4a68-a5c1-6d7e14108d95 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:46.406023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bff85-8abc-4a68-a5c1-6d7e14108d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:46.448058 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-696bff85-8abc-4a68-a5c1-6d7e14108d95 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:46.453271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-696bff85-8abc-4a68-a5c1-6d7e14108d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:46.670181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 853/1707] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:46.712897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:46.716671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:46.867860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:46.902844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338fd35-be32-4ebb-a0e5-d35be4fd928f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:46.918250 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b902be-e72e-4994-9513-5d50bc1e7108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:46.918250 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b902be-e72e-4994-9513-5d50bc1e7108 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:46.918250 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b902be-e72e-4994-9513-5d50bc1e7108 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:46.918250 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b902be-e72e-4994-9513-5d50bc1e7108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:46.943763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b902be-e72e-4994-9513-5d50bc1e7108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:46.945674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b902be-e72e-4994-9513-5d50bc1e7108 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:46.977147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 855/1708] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:48.520321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d803c-c3e0-4047-a874-f4f4ce72121b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:48.521837 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d803c-c3e0-4047-a874-f4f4ce72121b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:48.522157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d803c-c3e0-4047-a874-f4f4ce72121b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:48.525177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 854/1709] 10.222.0.43 () {54 vars in 912 bytes} [Mon Jul 4 08:03:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:03:48.530639 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a56bb4-2b60-482f-83e5-37da2263a9fe None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:48.531334 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a56bb4-2b60-482f-83e5-37da2263a9fe None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:48.531334 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a56bb4-2b60-482f-83e5-37da2263a9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:48.622777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-59a56bb4-2b60-482f-83e5-37da2263a9fe None None] MFA Rules not processed for user `429e173890b84ee289abc9793794ddb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:48.624767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-59a56bb4-2b60-482f-83e5-37da2263a9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:48.653909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 856/1710] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:03:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:48.682173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d803c-c3e0-4047-a874-f4f4ce72121b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:48.684597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8c9d803c-c3e0-4047-a874-f4f4ce72121b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:48.722808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d803c-c3e0-4047-a874-f4f4ce72121b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:48.739548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abc723-aa64-4e9f-a41e-ebfb8ba07c87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:48.744932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc723-aa64-4e9f-a41e-ebfb8ba07c87 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:48.753179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc723-aa64-4e9f-a41e-ebfb8ba07c87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:48.755328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc723-aa64-4e9f-a41e-ebfb8ba07c87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:48.767710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abc723-aa64-4e9f-a41e-ebfb8ba07c87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:48.769536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abc723-aa64-4e9f-a41e-ebfb8ba07c87 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:48.803174 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a580dc8-00c1-4a7d-a6c8-b8deb55500b7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:48.803530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a580dc8-00c1-4a7d-a6c8-b8deb55500b7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:48.804113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a580dc8-00c1-4a7d-a6c8-b8deb55500b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:48.804490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 855/1711] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:48.910785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5a580dc8-00c1-4a7d-a6c8-b8deb55500b7 None None] MFA Rules not processed for user `1501eb69e0a447ce83baa5bdee770d1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:48.912337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5a580dc8-00c1-4a7d-a6c8-b8deb55500b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:49.156439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 857/1712] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:03:49.173523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abc723-aa64-4e9f-a41e-ebfb8ba07c87 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:49.176498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-37abc723-aa64-4e9f-a41e-ebfb8ba07c87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:49.206173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eef9ce-7ea7-4bc8-b82d-621d7b201da0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:49.207001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eef9ce-7ea7-4bc8-b82d-621d7b201da0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:49.208687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eef9ce-7ea7-4bc8-b82d-621d7b201da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:49.308366 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-08eef9ce-7ea7-4bc8-b82d-621d7b201da0 None None] MFA Rules not processed for user `126ba607541e4c4895b47b4b7602ce01`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:49.317518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-08eef9ce-7ea7-4bc8-b82d-621d7b201da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:49.481238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abc723-aa64-4e9f-a41e-ebfb8ba07c87 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:49.495592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a43d48-d33c-4178-afc3-57c729ae712b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:49.497014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a43d48-d33c-4178-afc3-57c729ae712b service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:49.497707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a43d48-d33c-4178-afc3-57c729ae712b service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:49.498330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a43d48-d33c-4178-afc3-57c729ae712b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:49.552625 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a43d48-d33c-4178-afc3-57c729ae712b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:49.568563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a43d48-d33c-4178-afc3-57c729ae712b service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:49.626397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 856/1713] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:49.633197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a43d48-d33c-4178-afc3-57c729ae712b service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:49.642129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 858/1714] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:49.648412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b9a43d48-d33c-4178-afc3-57c729ae712b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:49.705981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a43d48-d33c-4178-afc3-57c729ae712b service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:49.735991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:49.743859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:49.746128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:49.747564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c admin admin] PATH_INFO: `/v3/projects/784a07c025314f0ebd6dc72bd77cc5d9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:49.754464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=784a07c025314f0ebd6dc72bd77cc5d9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:49.775666 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:50.010786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '784a07c025314f0ebd6dc72bd77cc5d9', 'request_id': 'req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:50.049410 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '784a07c025314f0ebd6dc72bd77cc5d9', 'request_id': 'req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:50.082711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '784a07c025314f0ebd6dc72bd77cc5d9', 'request_id': 'req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:50.093883 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:50.150077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eef9ce-7ea7-4bc8-b82d-621d7b201da0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:50.151719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-08eef9ce-7ea7-4bc8-b82d-621d7b201da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:50.168102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c admin admin] The token cache is being invalidate because project 784a07c025314f0ebd6dc72bd77cc5d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:03:50.168535 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:50.171335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5a25c9aa-87b8-4db5-8950-ba9343b43f2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:50.184731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 857/1715] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:03:49 2022] DELETE /identity/v3/projects/784a07c025314f0ebd6dc72bd77cc5d9 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:50.194466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307456a9-8d12-489d-a8ee-3d91212b4ebd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:50.195189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307456a9-8d12-489d-a8ee-3d91212b4ebd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:50.195488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307456a9-8d12-489d-a8ee-3d91212b4ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:50.225662 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eef9ce-7ea7-4bc8-b82d-621d7b201da0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:50.331576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-307456a9-8d12-489d-a8ee-3d91212b4ebd None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:50.333941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-307456a9-8d12-489d-a8ee-3d91212b4ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:50.594766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 858/1716] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:50.596274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ea62b-2e4a-468e-b0df-baa3b143eea4 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:50.596572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ea62b-2e4a-468e-b0df-baa3b143eea4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:50.596853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ea62b-2e4a-468e-b0df-baa3b143eea4 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:50.603756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 859/1717] 10.222.0.43 () {54 vars in 960 bytes} [Mon Jul 4 08:03:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:03:50.605374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf274071-000d-4ef3-8a3d-72ead70cfd03 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:50.605565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf274071-000d-4ef3-8a3d-72ead70cfd03 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:50.605930 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf274071-000d-4ef3-8a3d-72ead70cfd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:50.650928 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bf274071-000d-4ef3-8a3d-72ead70cfd03 None None] MFA Rules not processed for user `18b1072539af49729dff7ee9ea445c77`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:50.652422 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bf274071-000d-4ef3-8a3d-72ead70cfd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:50.701653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:50.756337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:50.765991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:50.769320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:50.770441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] PATH_INFO: `/v3/users/21aa08cb7bbc44928aa0e29085e7d44f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:50.780540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21aa08cb7bbc44928aa0e29085e7d44f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:50.795832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:50.858162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 860/1718] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:50.869131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43d231-dd91-4244-998d-60b22e19f61d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:50.869449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43d231-dd91-4244-998d-60b22e19f61d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:50.874481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43d231-dd91-4244-998d-60b22e19f61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:50.945867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0f43d231-dd91-4244-998d-60b22e19f61d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:50.949941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0f43d231-dd91-4244-998d-60b22e19f61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:50.971751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21aa08cb7bbc44928aa0e29085e7d44f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:50.983468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 861/1719] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:50.985308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6bcba-3b1d-4fa1-96f7-ba9411ade0d8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:50.985530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6bcba-3b1d-4fa1-96f7-ba9411ade0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:50.985915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6bcba-3b1d-4fa1-96f7-ba9411ade0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:50.995945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21aa08cb7bbc44928aa0e29085e7d44f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:51.014353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21aa08cb7bbc44928aa0e29085e7d44f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:51.022577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21aa08cb7bbc44928aa0e29085e7d44f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:51.056344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21aa08cb7bbc44928aa0e29085e7d44f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:51.069206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-24a6bcba-3b1d-4fa1-96f7-ba9411ade0d8 None None] MFA Rules not processed for user `f57dccfe9f5f4bc397f4c470fbf047dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:51.086491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-24a6bcba-3b1d-4fa1-96f7-ba9411ade0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:51.088400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 859/1720] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:03:50 2022] DELETE /identity/v3/users/21aa08cb7bbc44928aa0e29085e7d44f => generated 0 bytes in 939 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:03:51.096644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:51.105517 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:51.344436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 862/1721] 10.222.0.43 () {58 vars in 1057 bytes} [Mon Jul 4 08:03:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:51.355832 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f365e-051f-4533-8369-3719fa7bdd23 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:51.356120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f365e-051f-4533-8369-3719fa7bdd23 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:51.357374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f365e-051f-4533-8369-3719fa7bdd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:51.407359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e18f365e-051f-4533-8369-3719fa7bdd23 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:51.421517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e18f365e-051f-4533-8369-3719fa7bdd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:51.451058 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 863/1722] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:51.452606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f365e-051f-4533-8369-3719fa7bdd23 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:51.460299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018add8-4e80-4643-9762-66a66377f4e7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:51.482408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f365e-051f-4533-8369-3719fa7bdd23 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:51.486886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be7b8b-48c0-494f-a160-4081a21cbea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:51.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be7b8b-48c0-494f-a160-4081a21cbea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:51.488141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be7b8b-48c0-494f-a160-4081a21cbea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:51.488517 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be7b8b-48c0-494f-a160-4081a21cbea6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:51.489325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be7b8b-48c0-494f-a160-4081a21cbea6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:03:51.489673 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be7b8b-48c0-494f-a160-4081a21cbea6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:51.493834 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcf79f-02ee-4e2a-8936-c6b74a57afc4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57dccfe9f5f4bc397f4c470fbf047dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:51.495177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcf79f-02ee-4e2a-8936-c6b74a57afc4 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:51.495336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be7b8b-48c0-494f-a160-4081a21cbea6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:51.496283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcf79f-02ee-4e2a-8936-c6b74a57afc4 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:51.496792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcf79f-02ee-4e2a-8936-c6b74a57afc4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:51.500644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c8dcf79f-02ee-4e2a-8936-c6b74a57afc4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:51.530986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcf79f-02ee-4e2a-8936-c6b74a57afc4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:51.539492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcf79f-02ee-4e2a-8936-c6b74a57afc4 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:51.550027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 860/1723] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:03:51.552361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be7b8b-48c0-494f-a160-4081a21cbea6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:51.579842 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 864/1724] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:51.587446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be7b8b-48c0-494f-a160-4081a21cbea6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:51.588044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcf79f-02ee-4e2a-8936-c6b74a57afc4 service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:51.595278 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c8dcf79f-02ee-4e2a-8936-c6b74a57afc4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:51.618580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2d144-5997-4f56-9991-7dd43c0f3d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:51.619315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2d144-5997-4f56-9991-7dd43c0f3d85 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:51.619625 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2d144-5997-4f56-9991-7dd43c0f3d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:51.619900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2d144-5997-4f56-9991-7dd43c0f3d85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:51.620828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2d144-5997-4f56-9991-7dd43c0f3d85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:51.623152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2d144-5997-4f56-9991-7dd43c0f3d85 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:51.632954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:51.733540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 861/1725] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:51 2022] POST /identity/v3/projects => generated 363 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:51.744513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2d144-5997-4f56-9991-7dd43c0f3d85 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:51.786297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcf79f-02ee-4e2a-8936-c6b74a57afc4 service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:51.959100 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735e2df-40fb-4528-8ca1-5d86f01c05d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:51.960269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735e2df-40fb-4528-8ca1-5d86f01c05d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:51.960804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735e2df-40fb-4528-8ca1-5d86f01c05d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:51.961283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735e2df-40fb-4528-8ca1-5d86f01c05d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:51.980012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:52.079308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2d144-5997-4f56-9991-7dd43c0f3d85 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:52.101254 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff0d97-7246-4361-bdda-ed7cd3e44cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:52.102383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff0d97-7246-4361-bdda-ed7cd3e44cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:52.103639 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff0d97-7246-4361-bdda-ed7cd3e44cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:52.104159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff0d97-7246-4361-bdda-ed7cd3e44cd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:52.105245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff0d97-7246-4361-bdda-ed7cd3e44cd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:52.114016 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff0d97-7246-4361-bdda-ed7cd3e44cd4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:52.167558 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 862/1726] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:51 2022] POST /identity/v3/users => generated 282 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:52.168929 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff0d97-7246-4361-bdda-ed7cd3e44cd4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:52.196738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff0d97-7246-4361-bdda-ed7cd3e44cd4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:52.202443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735e2df-40fb-4528-8ca1-5d86f01c05d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:52.206668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735e2df-40fb-4528-8ca1-5d86f01c05d6 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:52.218733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4426fee-283c-4f20-a249-2ffda5714220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:52.220123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4426fee-283c-4f20-a249-2ffda5714220 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:52.220648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4426fee-283c-4f20-a249-2ffda5714220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:52.221183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4426fee-283c-4f20-a249-2ffda5714220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:52.226615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4426fee-283c-4f20-a249-2ffda5714220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:52.229701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4426fee-283c-4f20-a249-2ffda5714220 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:52.262648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 865/1727] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:52.263406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 863/1728] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:52.271780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735e2df-40fb-4528-8ca1-5d86f01c05d6 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:52.339478 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735e2df-40fb-4528-8ca1-5d86f01c05d6 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:52.368025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23128a1b-010b-4c4f-b1ef-6c416b23909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:52.371723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23128a1b-010b-4c4f-b1ef-6c416b23909d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:52.376732 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23128a1b-010b-4c4f-b1ef-6c416b23909d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:52.377569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23128a1b-010b-4c4f-b1ef-6c416b23909d admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:52.383095 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23128a1b-010b-4c4f-b1ef-6c416b23909d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:52.389893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23128a1b-010b-4c4f-b1ef-6c416b23909d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:52.457998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 866/1729] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:52.463323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4426fee-283c-4f20-a249-2ffda5714220 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:52.480317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4426fee-283c-4f20-a249-2ffda5714220 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:52.493775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36098125-c1f9-4a3e-829f-f460e450c07f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:52.494091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36098125-c1f9-4a3e-829f-f460e450c07f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:52.494906 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36098125-c1f9-4a3e-829f-f460e450c07f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:52.496547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 867/1730] 10.222.0.43 () {54 vars in 912 bytes} [Mon Jul 4 08:03:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 08:03:52.502906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8a144-86d7-4de8-a2a4-5b291549750e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:52.504729 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8a144-86d7-4de8-a2a4-5b291549750e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:52.505206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8a144-86d7-4de8-a2a4-5b291549750e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:52.505733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8a144-86d7-4de8-a2a4-5b291549750e admin admin] PATH_INFO: `/v3/projects/7d53faf2d06342519c161f042f4212a3/users/760f9bf6bd5743c09fd8e0d87b100272/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:52.519040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31d5d-c506-4670-b337-d700e8d122c2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:52.519409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31d5d-c506-4670-b337-d700e8d122c2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:52.519829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31d5d-c506-4670-b337-d700e8d122c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:52.593794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8a144-86d7-4de8-a2a4-5b291549750e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d53faf2d06342519c161f042f4212a3, user_id=760f9bf6bd5743c09fd8e0d87b100272, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:52.594256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d6d31d5d-c506-4670-b337-d700e8d122c2 None None] MFA Rules not processed for user `429e173890b84ee289abc9793794ddb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:52.601500 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8a144-86d7-4de8-a2a4-5b291549750e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:52.606081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d6d31d5d-c506-4670-b337-d700e8d122c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:52.660163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 868/1731] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:03:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:03:52.682344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 864/1732] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:52 2022] PUT /identity/v3/projects/7d53faf2d06342519c161f042f4212a3/users/760f9bf6bd5743c09fd8e0d87b100272/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:52.682676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d31d5d-c506-4670-b337-d700e8d122c2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:52.686240 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eca9f4-2b8b-4d3a-abb5-a2974e94c83a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:52.688105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eca9f4-2b8b-4d3a-abb5-a2974e94c83a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:52.688105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eca9f4-2b8b-4d3a-abb5-a2974e94c83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:52.688374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d31d5d-c506-4670-b337-d700e8d122c2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:52.773177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-20eca9f4-2b8b-4d3a-abb5-a2974e94c83a None None] MFA Rules not processed for user `760f9bf6bd5743c09fd8e0d87b100272`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:52.777132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-20eca9f4-2b8b-4d3a-abb5-a2974e94c83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:52.932416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56a204-1897-472b-907a-ae8eafa530d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:52.933944 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56a204-1897-472b-907a-ae8eafa530d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:52.939313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56a204-1897-472b-907a-ae8eafa530d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:52.939667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56a204-1897-472b-907a-ae8eafa530d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:52.961474 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56a204-1897-472b-907a-ae8eafa530d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:52.963080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56a204-1897-472b-907a-ae8eafa530d8 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:52.968444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:52.999173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 869/1733] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:03:53.012060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 865/1734] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:52 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:03:53.013798 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56a204-1897-472b-907a-ae8eafa530d8 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:53.264475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56a204-1897-472b-907a-ae8eafa530d8 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:53.285361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049b292-bfd2-41e3-9778-ccac44b888a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:53.289028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049b292-bfd2-41e3-9778-ccac44b888a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:53.289982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049b292-bfd2-41e3-9778-ccac44b888a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:53.290624 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049b292-bfd2-41e3-9778-ccac44b888a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:53.292684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049b292-bfd2-41e3-9778-ccac44b888a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:53.300187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049b292-bfd2-41e3-9778-ccac44b888a6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:53.414133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 870/1735] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:53 2022] POST /identity/v3/projects => generated 365 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:53.418539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eca9f4-2b8b-4d3a-abb5-a2974e94c83a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:53.447114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eca9f4-2b8b-4d3a-abb5-a2974e94c83a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:53.657171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049b292-bfd2-41e3-9778-ccac44b888a6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:53.659381 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9049b292-bfd2-41e3-9778-ccac44b888a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:53.742123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184176c-a850-49e5-998b-9fb46cf0edc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:53.742974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184176c-a850-49e5-998b-9fb46cf0edc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:53.743269 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184176c-a850-49e5-998b-9fb46cf0edc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:53.743852 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184176c-a850-49e5-998b-9fb46cf0edc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:53.744752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184176c-a850-49e5-998b-9fb46cf0edc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:53.747990 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184176c-a850-49e5-998b-9fb46cf0edc5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:53.817046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 866/1736] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:53 2022] POST /identity/v3/users => generated 289 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:53.819883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184176c-a850-49e5-998b-9fb46cf0edc5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:53.844798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184176c-a850-49e5-998b-9fb46cf0edc5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:53.875934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f238f96-5cfe-451a-aafb-02a43c2e9da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:53.876653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f238f96-5cfe-451a-aafb-02a43c2e9da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:53.877592 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f238f96-5cfe-451a-aafb-02a43c2e9da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:53.880519 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f238f96-5cfe-451a-aafb-02a43c2e9da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:53.881467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f238f96-5cfe-451a-aafb-02a43c2e9da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:53.883128 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f238f96-5cfe-451a-aafb-02a43c2e9da5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:53.916649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 867/1737] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:53.921539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f238f96-5cfe-451a-aafb-02a43c2e9da5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:53.926610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049b292-bfd2-41e3-9778-ccac44b888a6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:53.937474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f238f96-5cfe-451a-aafb-02a43c2e9da5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:53.949402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:53.951276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:53.951569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:53.951855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] PATH_INFO: `/v3/projects/0232db2998e5444987fe8886b870957e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:53.954489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8fe12-de73-4558-9983-5de8c8836caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:53.954824 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8fe12-de73-4558-9983-5de8c8836caf admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:53.955060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8fe12-de73-4558-9983-5de8c8836caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:53.955361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8fe12-de73-4558-9983-5de8c8836caf admin admin] PATH_INFO: `/v3/projects/8ac5a4c807724ef2924debbb8902b055/users/5927029f8f374dc5bea9834fc2ad6531/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:53.961053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0232db2998e5444987fe8886b870957e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:53.963086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:53.993338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:54.022098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8fe12-de73-4558-9983-5de8c8836caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac5a4c807724ef2924debbb8902b055, user_id=5927029f8f374dc5bea9834fc2ad6531, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:54.030163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8fe12-de73-4558-9983-5de8c8836caf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:54.068022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 868/1738] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:53 2022] PUT /identity/v3/projects/8ac5a4c807724ef2924debbb8902b055/users/5927029f8f374dc5bea9834fc2ad6531/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:03:54.073013 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8fe12-de73-4558-9983-5de8c8836caf admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:54.128129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0232db2998e5444987fe8886b870957e', 'request_id': 'req-2f5c01fb-b18c-43bb-8291-1c2948c62269'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:54.139645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0232db2998e5444987fe8886b870957e', 'request_id': 'req-2f5c01fb-b18c-43bb-8291-1c2948c62269'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:54.156855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0232db2998e5444987fe8886b870957e', 'request_id': 'req-2f5c01fb-b18c-43bb-8291-1c2948c62269'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:54.187885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] The token cache is being invalidate because project 0232db2998e5444987fe8886b870957e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:03:54.188248 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:54.189938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:54.193916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 871/1739] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:03:53 2022] DELETE /identity/v3/projects/0232db2998e5444987fe8886b870957e => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:54.200362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:54.207007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:54.259301 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:54.334860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8fe12-de73-4558-9983-5de8c8836caf admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:54.472999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c01fb-b18c-43bb-8291-1c2948c62269 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:54.496137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:54.497551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:54.498946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:54.499791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] PATH_INFO: `/v3/users/154ea0e85a124e6da43a676744b5eeb4` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:54.502567 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=154ea0e85a124e6da43a676744b5eeb4)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:54.508887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:54.629429 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb4620-75b7-4e21-985c-c61f76420243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:54.632827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb4620-75b7-4e21-985c-c61f76420243 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:54.633582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb4620-75b7-4e21-985c-c61f76420243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:54.633904 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb4620-75b7-4e21-985c-c61f76420243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:54.636160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb4620-75b7-4e21-985c-c61f76420243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:54.640383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb4620-75b7-4e21-985c-c61f76420243 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:54.663620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154ea0e85a124e6da43a676744b5eeb4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:54.664172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154ea0e85a124e6da43a676744b5eeb4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:54.664824 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154ea0e85a124e6da43a676744b5eeb4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:54.674694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154ea0e85a124e6da43a676744b5eeb4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:54.689169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 869/1740] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 615 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:54.693263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb4620-75b7-4e21-985c-c61f76420243 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:54.697905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0abb4620-75b7-4e21-985c-c61f76420243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:54.716932 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb4620-75b7-4e21-985c-c61f76420243 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:54.723617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635cf9e-89c7-4acf-9a5e-8d6f17494ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:54.726101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635cf9e-89c7-4acf-9a5e-8d6f17494ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:54.726443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635cf9e-89c7-4acf-9a5e-8d6f17494ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:54.727221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635cf9e-89c7-4acf-9a5e-8d6f17494ae1 admin admin] PATH_INFO: `/v3/projects/8ac5a4c807724ef2924debbb8902b055/users/5927029f8f374dc5bea9834fc2ad6531/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:54.730777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635cf9e-89c7-4acf-9a5e-8d6f17494ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac5a4c807724ef2924debbb8902b055, user_id=5927029f8f374dc5bea9834fc2ad6531, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:54.744194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635cf9e-89c7-4acf-9a5e-8d6f17494ae1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:54.756664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154ea0e85a124e6da43a676744b5eeb4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:54.771622 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 870/1741] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:54 2022] PUT /identity/v3/projects/8ac5a4c807724ef2924debbb8902b055/users/5927029f8f374dc5bea9834fc2ad6531/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:54.784647 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64490b-292e-4ba4-98a5-cd126331d197 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:54.785422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64490b-292e-4ba4-98a5-cd126331d197 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:54.786288 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64490b-292e-4ba4-98a5-cd126331d197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:54.801505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 872/1742] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:03:54 2022] DELETE /identity/v3/users/154ea0e85a124e6da43a676744b5eeb4 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:54.874963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fe64490b-292e-4ba4-98a5-cd126331d197 None None] MFA Rules not processed for user `5927029f8f374dc5bea9834fc2ad6531`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:54.878729 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fe64490b-292e-4ba4-98a5-cd126331d197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:55.369857 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 871/1743] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:54 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 591 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:55.410421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:55.412727 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:55.564723 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64490b-292e-4ba4-98a5-cd126331d197 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:55.570150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fe64490b-292e-4ba4-98a5-cd126331d197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:55.591294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:55.719118 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b693ae2-61ab-4cf1-a34b-c163cca465ca admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:55.730589 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cade196-f305-4471-8883-eaeb907d16b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:55.731750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cade196-f305-4471-8883-eaeb907d16b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:55.732106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cade196-f305-4471-8883-eaeb907d16b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:55.732614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cade196-f305-4471-8883-eaeb907d16b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:55.782901 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cade196-f305-4471-8883-eaeb907d16b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:55.817640 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cade196-f305-4471-8883-eaeb907d16b7 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:55.878172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 873/1744] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:03:55.939309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64490b-292e-4ba4-98a5-cd126331d197 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:55.954505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083d9ba-30d6-4421-bf09-29cd519559f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:55.955591 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083d9ba-30d6-4421-bf09-29cd519559f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:55.955979 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083d9ba-30d6-4421-bf09-29cd519559f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:55.956467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083d9ba-30d6-4421-bf09-29cd519559f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:55.966860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4083d9ba-30d6-4421-bf09-29cd519559f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:55.976968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cade196-f305-4471-8883-eaeb907d16b7 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:55.979370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7cade196-f305-4471-8883-eaeb907d16b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:56.165751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083d9ba-30d6-4421-bf09-29cd519559f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:56.167519 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083d9ba-30d6-4421-bf09-29cd519559f0 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:56.210291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 872/1745] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:03:56.239615 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:56.373275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52749b82-d177-4603-811d-833f1796c0ae None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:56.373709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52749b82-d177-4603-811d-833f1796c0ae None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:56.375317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52749b82-d177-4603-811d-833f1796c0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:56.375931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cade196-f305-4471-8883-eaeb907d16b7 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:56.408039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-52749b82-d177-4603-811d-833f1796c0ae None None] MFA Rules not processed for user `1501eb69e0a447ce83baa5bdee770d1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:56.411390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-52749b82-d177-4603-811d-833f1796c0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:56.436293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a39e3e-bd59-4036-9c77-a1c679369eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:56.440201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a39e3e-bd59-4036-9c77-a1c679369eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:56.452004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a39e3e-bd59-4036-9c77-a1c679369eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:56.452512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a39e3e-bd59-4036-9c77-a1c679369eb1 admin admin] PATH_INFO: `/v3/projects/ab316665fbb942fe8858d99acb2622f4` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:56.455348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a39e3e-bd59-4036-9c77-a1c679369eb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab316665fbb942fe8858d99acb2622f4)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:56.460145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a39e3e-bd59-4036-9c77-a1c679369eb1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:56.580930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 873/1746] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:56.848131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a7a39e3e-bd59-4036-9c77-a1c679369eb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab316665fbb942fe8858d99acb2622f4', 'request_id': 'req-a7a39e3e-bd59-4036-9c77-a1c679369eb1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:56.876782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a7a39e3e-bd59-4036-9c77-a1c679369eb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab316665fbb942fe8858d99acb2622f4', 'request_id': 'req-a7a39e3e-bd59-4036-9c77-a1c679369eb1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:56.937217 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a7a39e3e-bd59-4036-9c77-a1c679369eb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab316665fbb942fe8858d99acb2622f4', 'request_id': 'req-a7a39e3e-bd59-4036-9c77-a1c679369eb1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:56.986175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a7a39e3e-bd59-4036-9c77-a1c679369eb1 admin admin] The token cache is being invalidate because project ab316665fbb942fe8858d99acb2622f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:03:56.988988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a7a39e3e-bd59-4036-9c77-a1c679369eb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:56.991555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a7a39e3e-bd59-4036-9c77-a1c679369eb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:57.008856 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b23a22-2ebd-4931-a1ad-6c199bcaf9ea None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:57.009808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b23a22-2ebd-4931-a1ad-6c199bcaf9ea None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:57.009808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b23a22-2ebd-4931-a1ad-6c199bcaf9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:57.022554 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 874/1747] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:03:55 2022] DELETE /identity/v3/projects/ab316665fbb942fe8858d99acb2622f4 => generated 0 bytes in 1027 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:03:57.075989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c9b23a22-2ebd-4931-a1ad-6c199bcaf9ea None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:57.083578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c9b23a22-2ebd-4931-a1ad-6c199bcaf9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:57.381235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:57.448354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 874/1748] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:57.450158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc5827-9c89-4655-bc4c-1d84fcf77425 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:57.450571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc5827-9c89-4655-bc4c-1d84fcf77425 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:57.450986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc5827-9c89-4655-bc4c-1d84fcf77425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:57.543932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bedc5827-9c89-4655-bc4c-1d84fcf77425 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:57.551166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bedc5827-9c89-4655-bc4c-1d84fcf77425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:57.598628 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 875/1749] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:03:57.603470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b23a22-2ebd-4931-a1ad-6c199bcaf9ea None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:57.629442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b23a22-2ebd-4931-a1ad-6c199bcaf9ea None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:57.646601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc396f9-0850-459f-8b5f-e4f368b89f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:57.648865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc396f9-0850-459f-8b5f-e4f368b89f69 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:57.649149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc396f9-0850-459f-8b5f-e4f368b89f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:57.651615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc396f9-0850-459f-8b5f-e4f368b89f69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:57.657308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc396f9-0850-459f-8b5f-e4f368b89f69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:03:57.658435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc396f9-0850-459f-8b5f-e4f368b89f69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:57.665567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc396f9-0850-459f-8b5f-e4f368b89f69 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:57.744358 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 875/1750] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:03:57.751445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc5827-9c89-4655-bc4c-1d84fcf77425 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:57.777562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc5827-9c89-4655-bc4c-1d84fcf77425 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:57.823816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc656e-593b-465d-88ba-cc8913e72736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:57.833281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc656e-593b-465d-88ba-cc8913e72736 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:57.833827 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc656e-593b-465d-88ba-cc8913e72736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:57.834320 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc656e-593b-465d-88ba-cc8913e72736 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:57.853069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc656e-593b-465d-88ba-cc8913e72736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:57.863052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc656e-593b-465d-88ba-cc8913e72736 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:57.972163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 876/1751] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:57 2022] POST /identity/v3/projects => generated 399 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:57.976838 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc396f9-0850-459f-8b5f-e4f368b89f69 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:57.990524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc396f9-0850-459f-8b5f-e4f368b89f69 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:58.215499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac4e17-a873-4934-87c9-ca35c3c36459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:58.217434 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac4e17-a873-4934-87c9-ca35c3c36459 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:58.219834 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac4e17-a873-4934-87c9-ca35c3c36459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:58.220398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac4e17-a873-4934-87c9-ca35c3c36459 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:58.222523 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac4e17-a873-4934-87c9-ca35c3c36459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:58.232470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac4e17-a873-4934-87c9-ca35c3c36459 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:58.314957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc656e-593b-465d-88ba-cc8913e72736 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:58.324535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 876/1752] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:57 2022] POST /identity/v3/users => generated 300 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:58.331023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc656e-593b-465d-88ba-cc8913e72736 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:58.339908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379dd3cb-1305-429f-ad53-c69eba51c221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:58.340705 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dd3cb-1305-429f-ad53-c69eba51c221 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:58.341179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dd3cb-1305-429f-ad53-c69eba51c221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:58.341558 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dd3cb-1305-429f-ad53-c69eba51c221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:58.342510 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379dd3cb-1305-429f-ad53-c69eba51c221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:58.353241 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379dd3cb-1305-429f-ad53-c69eba51c221 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:58.395387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac4e17-a873-4934-87c9-ca35c3c36459 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:58.413362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac4e17-a873-4934-87c9-ca35c3c36459 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:58.414251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 877/1753] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:03:58.436006 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcf884-ecc4-4de3-82ff-a81990096aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:58.436902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcf884-ecc4-4de3-82ff-a81990096aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:58.437247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcf884-ecc4-4de3-82ff-a81990096aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:58.437534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcf884-ecc4-4de3-82ff-a81990096aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:58.438345 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcf884-ecc4-4de3-82ff-a81990096aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:58.440060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcf884-ecc4-4de3-82ff-a81990096aa0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:58.497320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 877/1754] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:03:58.504608 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc56715-d586-4bd1-a632-ae5e5d464743 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:58.504934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc56715-d586-4bd1-a632-ae5e5d464743 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:58.505297 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc56715-d586-4bd1-a632-ae5e5d464743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:58.505884 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379dd3cb-1305-429f-ad53-c69eba51c221 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:58.561269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379dd3cb-1305-429f-ad53-c69eba51c221 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:58.578806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7bc56715-d586-4bd1-a632-ae5e5d464743 None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:58.589648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2ed7f-372c-4a6c-9e34-a802d791a7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:58.590635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7bc56715-d586-4bd1-a632-ae5e5d464743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:58.595895 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2ed7f-372c-4a6c-9e34-a802d791a7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:58.596683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2ed7f-372c-4a6c-9e34-a802d791a7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:58.599369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2ed7f-372c-4a6c-9e34-a802d791a7c7 admin admin] PATH_INFO: `/v3/projects/c3c1c96638294aad8b0d2c80b7f08499/users/6f98c5fb5cec43f3b0de7b796bc5fa37/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:58.679173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2ed7f-372c-4a6c-9e34-a802d791a7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c1c96638294aad8b0d2c80b7f08499, user_id=6f98c5fb5cec43f3b0de7b796bc5fa37, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:58.704713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2ed7f-372c-4a6c-9e34-a802d791a7c7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:58.736676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 878/1755] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:03:58 2022] PUT /identity/v3/projects/c3c1c96638294aad8b0d2c80b7f08499/users/6f98c5fb5cec43f3b0de7b796bc5fa37/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:58.742821 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2ed7f-372c-4a6c-9e34-a802d791a7c7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:58.753904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f0a2ed7f-372c-4a6c-9e34-a802d791a7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:58.790671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 878/1756] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:03:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:58.807631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2d259-3b19-47dc-b181-bc8dee348130 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:58.807631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2d259-3b19-47dc-b181-bc8dee348130 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:58.807631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2d259-3b19-47dc-b181-bc8dee348130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:58.899625 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-50b2d259-3b19-47dc-b181-bc8dee348130 None None] MFA Rules not processed for user `6f98c5fb5cec43f3b0de7b796bc5fa37`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:58.903804 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-50b2d259-3b19-47dc-b181-bc8dee348130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:58.990113 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2ed7f-372c-4a6c-9e34-a802d791a7c7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:59.009443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:59.015068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:59.015542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:59.015842 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] PATH_INFO: `/v3/users/760f9bf6bd5743c09fd8e0d87b100272` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:59.018279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] RBAC: Authorizing `identity:delete_user(user_id=760f9bf6bd5743c09fd8e0d87b100272)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:59.028055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:59.036340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:59.131974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:59.138580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '760f9bf6bd5743c09fd8e0d87b100272'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:59.140144 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '760f9bf6bd5743c09fd8e0d87b100272'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:59.141361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '760f9bf6bd5743c09fd8e0d87b100272'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:59.155033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '760f9bf6bd5743c09fd8e0d87b100272'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:59.157124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 879/1757] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:59.159000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2d259-3b19-47dc-b181-bc8dee348130 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:59.166923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-50b2d259-3b19-47dc-b181-bc8dee348130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:59.180318 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '760f9bf6bd5743c09fd8e0d87b100272'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:03:59.214089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 879/1758] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:03:58 2022] DELETE /identity/v3/users/760f9bf6bd5743c09fd8e0d87b100272 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:03:59.215520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:59.222487 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:59.361693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf2ede-f2a9-40ce-9764-b4f25e5c286e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:59.377072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f71de-ec67-46c3-803a-b3c5d0e825fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:59.379512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f71de-ec67-46c3-803a-b3c5d0e825fa admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:59.379512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f71de-ec67-46c3-803a-b3c5d0e825fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:59.380348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f71de-ec67-46c3-803a-b3c5d0e825fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:59.382052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f71de-ec67-46c3-803a-b3c5d0e825fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:59.392075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f71de-ec67-46c3-803a-b3c5d0e825fa admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:59.412125 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2d259-3b19-47dc-b181-bc8dee348130 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:59.470188 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 880/1759] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:03:59 2022] POST /identity/v3/projects => generated 401 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:59.473982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575d9f4-e545-4600-9b30-8e0e80100cd7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:59.474304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575d9f4-e545-4600-9b30-8e0e80100cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:59.474588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575d9f4-e545-4600-9b30-8e0e80100cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:59.522991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8575d9f4-e545-4600-9b30-8e0e80100cd7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:03:59.524530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8575d9f4-e545-4600-9b30-8e0e80100cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:59.642904 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f3ee6-3d39-4916-bc86-0ae4a2264dcc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:59.643627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f3ee6-3d39-4916-bc86-0ae4a2264dcc service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:59.643909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f3ee6-3d39-4916-bc86-0ae4a2264dcc service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:59.644187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f3ee6-3d39-4916-bc86-0ae4a2264dcc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:59.648931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f77f3ee6-3d39-4916-bc86-0ae4a2264dcc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:59.764088 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 881/1760] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:03:59.766110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575d9f4-e545-4600-9b30-8e0e80100cd7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:59.797998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575d9f4-e545-4600-9b30-8e0e80100cd7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:59.808519 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da25752-2fbd-495c-a3ff-504c203a86ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:59.811048 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da25752-2fbd-495c-a3ff-504c203a86ee admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:59.811360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da25752-2fbd-495c-a3ff-504c203a86ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:59.811843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da25752-2fbd-495c-a3ff-504c203a86ee admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:59.824191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da25752-2fbd-495c-a3ff-504c203a86ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:59.824874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f3ee6-3d39-4916-bc86-0ae4a2264dcc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:59.827048 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f3ee6-3d39-4916-bc86-0ae4a2264dcc service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:59.840492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da25752-2fbd-495c-a3ff-504c203a86ee admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:03:59.852249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:03:59.881670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 880/1761] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:03:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 08:03:59.883843 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f3ee6-3d39-4916-bc86-0ae4a2264dcc service swift] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:59.886207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f77f3ee6-3d39-4916-bc86-0ae4a2264dcc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:03:59.936299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 882/1762] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:03:59 2022] POST /identity/v3/users => generated 307 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:03:59.943230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da25752-2fbd-495c-a3ff-504c203a86ee admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:03:59.971546 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da25752-2fbd-495c-a3ff-504c203a86ee admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:03:59.990222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c67cff-33fc-4f86-aa0f-61d61d9dc9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:03:59.990963 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c67cff-33fc-4f86-aa0f-61d61d9dc9ab admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:03:59.991260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c67cff-33fc-4f86-aa0f-61d61d9dc9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:03:59.991538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c67cff-33fc-4f86-aa0f-61d61d9dc9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:03:59.993985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c67cff-33fc-4f86-aa0f-61d61d9dc9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:03:59.997021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c67cff-33fc-4f86-aa0f-61d61d9dc9ab admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:00.054594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:00.085544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f3ee6-3d39-4916-bc86-0ae4a2264dcc service swift] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:00.097382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 883/1763] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:00.099014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c67cff-33fc-4f86-aa0f-61d61d9dc9ab admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:00.175573 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c67cff-33fc-4f86-aa0f-61d61d9dc9ab admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:00.199154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e07c6-cb7f-4629-8c27-5891ae22e439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:00.200412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e07c6-cb7f-4629-8c27-5891ae22e439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:00.200700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e07c6-cb7f-4629-8c27-5891ae22e439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:00.200966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e07c6-cb7f-4629-8c27-5891ae22e439 admin admin] PATH_INFO: `/v3/projects/45d5f7dae38b4583930d78d695304eb4/users/0d94dad8396941ea9f18144575e303bc/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:00.304909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bfe76-5698-46fc-be48-d4da38217227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:00.305689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bfe76-5698-46fc-be48-d4da38217227 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:00.305989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bfe76-5698-46fc-be48-d4da38217227 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:00.306272 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bfe76-5698-46fc-be48-d4da38217227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:00.319505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e07c6-cb7f-4629-8c27-5891ae22e439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d5f7dae38b4583930d78d695304eb4, user_id=0d94dad8396941ea9f18144575e303bc, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:00.353819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e07c6-cb7f-4629-8c27-5891ae22e439 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:00.359382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bfe76-5698-46fc-be48-d4da38217227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:00.361566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bfe76-5698-46fc-be48-d4da38217227 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:00.393859 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 881/1764] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:03:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:00.457880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 884/1765] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:00 2022] PUT /identity/v3/projects/45d5f7dae38b4583930d78d695304eb4/users/0d94dad8396941ea9f18144575e303bc/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:00.460009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bfe76-5698-46fc-be48-d4da38217227 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:00.486644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bfe76-5698-46fc-be48-d4da38217227 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:00.735849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8274850-713b-4538-9109-38972c768f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:00.736799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8274850-713b-4538-9109-38972c768f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:00.737088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8274850-713b-4538-9109-38972c768f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:00.737465 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8274850-713b-4538-9109-38972c768f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:00.738362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8274850-713b-4538-9109-38972c768f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:00.741435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8274850-713b-4538-9109-38972c768f29 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:00.764765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 882/1766] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:00.769073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e07c6-cb7f-4629-8c27-5891ae22e439 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:00.790711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e07c6-cb7f-4629-8c27-5891ae22e439 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:00.798153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4aa3be-df52-4690-9a2a-1fb9ffb49ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:00.798981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aa3be-df52-4690-9a2a-1fb9ffb49ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:00.799545 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aa3be-df52-4690-9a2a-1fb9ffb49ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:00.800223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aa3be-df52-4690-9a2a-1fb9ffb49ae0 admin admin] PATH_INFO: `/v3/projects/45d5f7dae38b4583930d78d695304eb4/users/0d94dad8396941ea9f18144575e303bc/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:00.804107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4aa3be-df52-4690-9a2a-1fb9ffb49ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d5f7dae38b4583930d78d695304eb4, user_id=0d94dad8396941ea9f18144575e303bc, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:00.807908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4aa3be-df52-4690-9a2a-1fb9ffb49ae0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:00.827461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 885/1767] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:00 2022] PUT /identity/v3/projects/45d5f7dae38b4583930d78d695304eb4/users/0d94dad8396941ea9f18144575e303bc/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:00.839666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a00d5-54b8-462e-a4ad-bee52a675af0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:00.840079 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a00d5-54b8-462e-a4ad-bee52a675af0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:00.840264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a00d5-54b8-462e-a4ad-bee52a675af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:00.896678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e94a00d5-54b8-462e-a4ad-bee52a675af0 None None] MFA Rules not processed for user `0d94dad8396941ea9f18144575e303bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:00.925893 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e94a00d5-54b8-462e-a4ad-bee52a675af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:01.281778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4aa3be-df52-4690-9a2a-1fb9ffb49ae0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:01.287799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cf4aa3be-df52-4690-9a2a-1fb9ffb49ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:01.315530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 883/1768] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:00 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:01.332094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a00d5-54b8-462e-a4ad-bee52a675af0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:01.334542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e94a00d5-54b8-462e-a4ad-bee52a675af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:01.467989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:01.590906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a00d5-54b8-462e-a4ad-bee52a675af0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:01.631374 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40ef18-0a18-491a-8d5d-20995b6c83b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:01.632293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40ef18-0a18-491a-8d5d-20995b6c83b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:01.632682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40ef18-0a18-491a-8d5d-20995b6c83b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:01.632866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40ef18-0a18-491a-8d5d-20995b6c83b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:01.690868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40ef18-0a18-491a-8d5d-20995b6c83b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:01.697584 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40ef18-0a18-491a-8d5d-20995b6c83b9 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:01.757053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4aa3be-df52-4690-9a2a-1fb9ffb49ae0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:01.791922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:01.792831 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:01.793119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:01.793395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] PATH_INFO: `/v3/projects/7d53faf2d06342519c161f042f4212a3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:01.811566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d53faf2d06342519c161f042f4212a3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:01.829522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:01.832184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 884/1769] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:01 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:04:01.843624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb1ef3-6e46-4121-931c-21b826f4a074 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:01.849121 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb1ef3-6e46-4121-931c-21b826f4a074 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:01.852897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb1ef3-6e46-4121-931c-21b826f4a074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:02.053163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-30eb1ef3-6e46-4121-931c-21b826f4a074 None None] MFA Rules not processed for user `429e173890b84ee289abc9793794ddb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:02.058865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d53faf2d06342519c161f042f4212a3', 'request_id': 'req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:02.065741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-30eb1ef3-6e46-4121-931c-21b826f4a074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:02.092557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d53faf2d06342519c161f042f4212a3', 'request_id': 'req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:02.137667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d53faf2d06342519c161f042f4212a3', 'request_id': 'req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:02.180638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] The token cache is being invalidate because project 7d53faf2d06342519c161f042f4212a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:02.180932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:02.183269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:02.211577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 886/1770] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:01 2022] DELETE /identity/v3/projects/7d53faf2d06342519c161f042f4212a3 => generated 0 bytes in 931 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:02.212578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:02.219191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:02.406870 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 885/1771] 10.222.0.61 () {58 vars in 1015 bytes} [Mon Jul 4 08:04:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 573 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:02.407943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=6,keystone:SELECT=87,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:02.416933 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb1ef3-6e46-4121-931c-21b826f4a074 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:02.427213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-30eb1ef3-6e46-4121-931c-21b826f4a074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:02.494053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cdad2c-36ac-41ba-9de6-b0c70df06fd3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:02.508204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af8b4c9-3d42-4d6c-9f06-4a9d369faace service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:02.509449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8b4c9-3d42-4d6c-9f06-4a9d369faace service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:02.511633 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8b4c9-3d42-4d6c-9f06-4a9d369faace service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:02.513660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8b4c9-3d42-4d6c-9f06-4a9d369faace service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:02.520365 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9af8b4c9-3d42-4d6c-9f06-4a9d369faace service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:02.717867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb1ef3-6e46-4121-931c-21b826f4a074 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:02.743378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:02.744919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:02.745435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:02.745891 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] PATH_INFO: `/v3/users/5927029f8f374dc5bea9834fc2ad6531` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:02.749994 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5927029f8f374dc5bea9834fc2ad6531)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:02.752575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:02.817172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af8b4c9-3d42-4d6c-9f06-4a9d369faace service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:02.824015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af8b4c9-3d42-4d6c-9f06-4a9d369faace service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:02.884693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 887/1772] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:02.885908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af8b4c9-3d42-4d6c-9f06-4a9d369faace service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:02.906792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9af8b4c9-3d42-4d6c-9f06-4a9d369faace service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:02.972860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5927029f8f374dc5bea9834fc2ad6531'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:02.980997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af8b4c9-3d42-4d6c-9f06-4a9d369faace service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:02.993655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba82d6-19ef-45f7-9476-5b99832f97eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:02.997511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5927029f8f374dc5bea9834fc2ad6531'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:03.000599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba82d6-19ef-45f7-9476-5b99832f97eb service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:03.001689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba82d6-19ef-45f7-9476-5b99832f97eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:03.003507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba82d6-19ef-45f7-9476-5b99832f97eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:03.035274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5927029f8f374dc5bea9834fc2ad6531'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:03.036154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5927029f8f374dc5bea9834fc2ad6531'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:03.041729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba82d6-19ef-45f7-9476-5b99832f97eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:03.051354 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba82d6-19ef-45f7-9476-5b99832f97eb service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:03.090301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:03.148237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 888/1773] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:03.159974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5927029f8f374dc5bea9834fc2ad6531'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:03.189326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 886/1774] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:02 2022] DELETE /identity/v3/users/5927029f8f374dc5bea9834fc2ad6531 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:05.949501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba82d6-19ef-45f7-9476-5b99832f97eb service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:05.952805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b7ba82d6-19ef-45f7-9476-5b99832f97eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:06.005075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba82d6-19ef-45f7-9476-5b99832f97eb service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:06.075170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:06.077456 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:06.501986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84665c1-75cb-42fd-a5a7-01dca5df6640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:06.505677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84665c1-75cb-42fd-a5a7-01dca5df6640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:06.507367 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84665c1-75cb-42fd-a5a7-01dca5df6640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:06.508497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84665c1-75cb-42fd-a5a7-01dca5df6640 admin admin] PATH_INFO: `/v3/users/cfbaae1c8e8f4a3984bc169da9170515` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:06.514844 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84665c1-75cb-42fd-a5a7-01dca5df6640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfbaae1c8e8f4a3984bc169da9170515)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:06.522398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84665c1-75cb-42fd-a5a7-01dca5df6640 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:06.694195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5a0a4-b00d-4803-bdbe-c3f1e0afb7e3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:06.723406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ced6d9-8da3-486d-a317-25859b51a103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:06.725435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ced6d9-8da3-486d-a317-25859b51a103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:06.730973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ced6d9-8da3-486d-a317-25859b51a103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:06.731562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ced6d9-8da3-486d-a317-25859b51a103 admin admin] PATH_INFO: `/v3/projects/8ac5a4c807724ef2924debbb8902b055` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:06.733556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ced6d9-8da3-486d-a317-25859b51a103 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac5a4c807724ef2924debbb8902b055)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:06.735883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ced6d9-8da3-486d-a317-25859b51a103 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:06.834518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f84665c1-75cb-42fd-a5a7-01dca5df6640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfbaae1c8e8f4a3984bc169da9170515'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:06.837597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f84665c1-75cb-42fd-a5a7-01dca5df6640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfbaae1c8e8f4a3984bc169da9170515'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:06.841447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f84665c1-75cb-42fd-a5a7-01dca5df6640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfbaae1c8e8f4a3984bc169da9170515'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:06.896293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f84665c1-75cb-42fd-a5a7-01dca5df6640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfbaae1c8e8f4a3984bc169da9170515'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:06.966429 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-76ced6d9-8da3-486d-a317-25859b51a103 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac5a4c807724ef2924debbb8902b055', 'request_id': 'req-76ced6d9-8da3-486d-a317-25859b51a103'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:06.993211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f84665c1-75cb-42fd-a5a7-01dca5df6640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfbaae1c8e8f4a3984bc169da9170515'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:06.996007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-76ced6d9-8da3-486d-a317-25859b51a103 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac5a4c807724ef2924debbb8902b055', 'request_id': 'req-76ced6d9-8da3-486d-a317-25859b51a103'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:07.001027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:07.042277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-76ced6d9-8da3-486d-a317-25859b51a103 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac5a4c807724ef2924debbb8902b055', 'request_id': 'req-76ced6d9-8da3-486d-a317-25859b51a103'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:07.051336 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 889/1775] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:05 2022] DELETE /identity/v3/users/cfbaae1c8e8f4a3984bc169da9170515 => generated 0 bytes in 1098 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:07.058846 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05711195-7b63-4dea-ba35-dd9a379fa5dd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:07.059363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05711195-7b63-4dea-ba35-dd9a379fa5dd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:07.062335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05711195-7b63-4dea-ba35-dd9a379fa5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:07.082741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-76ced6d9-8da3-486d-a317-25859b51a103 admin admin] The token cache is being invalidate because project 8ac5a4c807724ef2924debbb8902b055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:07.084454 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-76ced6d9-8da3-486d-a317-25859b51a103 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:07.085462 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-76ced6d9-8da3-486d-a317-25859b51a103 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:07.087626 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 887/1776] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:06 2022] DELETE /identity/v3/projects/8ac5a4c807724ef2924debbb8902b055 => generated 0 bytes in 1015 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:07.095844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64cfd9-6916-40d0-bc76-cbeade379849 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:07.096708 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64cfd9-6916-40d0-bc76-cbeade379849 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:07.097223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64cfd9-6916-40d0-bc76-cbeade379849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:07.143403 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ab64cfd9-6916-40d0-bc76-cbeade379849 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:07.145316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ab64cfd9-6916-40d0-bc76-cbeade379849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:07.148677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-05711195-7b63-4dea-ba35-dd9a379fa5dd None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:07.157878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-05711195-7b63-4dea-ba35-dd9a379fa5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:07.413369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 888/1777] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:07.429752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa729f9b-05eb-43e2-91a3-40fbfd43de67 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:07.430649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa729f9b-05eb-43e2-91a3-40fbfd43de67 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:07.432596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa729f9b-05eb-43e2-91a3-40fbfd43de67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:07.452116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 890/1778] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:07.543625 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-aa729f9b-05eb-43e2-91a3-40fbfd43de67 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:07.548283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aa729f9b-05eb-43e2-91a3-40fbfd43de67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:07.552533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05711195-7b63-4dea-ba35-dd9a379fa5dd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:07.555547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-05711195-7b63-4dea-ba35-dd9a379fa5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:07.598117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 889/1779] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:07.603107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa729f9b-05eb-43e2-91a3-40fbfd43de67 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:07.631826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa729f9b-05eb-43e2-91a3-40fbfd43de67 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:07.650651 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05711195-7b63-4dea-ba35-dd9a379fa5dd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:07.674667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d749ccf-56e6-4890-a42f-86bdb43121f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:07.676037 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:07.677145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:07.677543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:07.678631 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] PATH_INFO: `/v3/users/9ac0a3aad50d4f4480ff361980521b28` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:07.678988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d749ccf-56e6-4890-a42f-86bdb43121f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:07.680363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d749ccf-56e6-4890-a42f-86bdb43121f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:07.680782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d749ccf-56e6-4890-a42f-86bdb43121f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:07.681977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ac0a3aad50d4f4480ff361980521b28)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:07.683173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d749ccf-56e6-4890-a42f-86bdb43121f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:04:07.687096 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d749ccf-56e6-4890-a42f-86bdb43121f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:07.692195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d749ccf-56e6-4890-a42f-86bdb43121f6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:07.692987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:07.745486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 890/1780] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:04:07.747089 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d749ccf-56e6-4890-a42f-86bdb43121f6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:07.780417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:07.812262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d749ccf-56e6-4890-a42f-86bdb43121f6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:07.830540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845d27b-421b-46b8-947c-8a4d5a82241f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:07.840091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d27b-421b-46b8-947c-8a4d5a82241f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:07.842194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d27b-421b-46b8-947c-8a4d5a82241f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:07.843759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d27b-421b-46b8-947c-8a4d5a82241f admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:07.847153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845d27b-421b-46b8-947c-8a4d5a82241f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:07.858939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845d27b-421b-46b8-947c-8a4d5a82241f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:07.904811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ac0a3aad50d4f4480ff361980521b28'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:07.907635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ac0a3aad50d4f4480ff361980521b28'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:07.910076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ac0a3aad50d4f4480ff361980521b28'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:07.953154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ac0a3aad50d4f4480ff361980521b28'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:08.000626 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 891/1781] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:07 2022] POST /identity/v3/projects => generated 365 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:08.004518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845d27b-421b-46b8-947c-8a4d5a82241f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:08.010320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7845d27b-421b-46b8-947c-8a4d5a82241f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:08.010627 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ac0a3aad50d4f4480ff361980521b28'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:08.053125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 891/1782] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:07 2022] DELETE /identity/v3/users/9ac0a3aad50d4f4480ff361980521b28 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:08.054148 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:08.061222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:08.242622 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845d27b-421b-46b8-947c-8a4d5a82241f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:08.305282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d36395-9b28-4ffc-8f61-d6bc4499803b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:08.321198 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bece32-1bd6-46e9-83a9-316785d07043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:08.323623 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bece32-1bd6-46e9-83a9-316785d07043 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:08.323953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bece32-1bd6-46e9-83a9-316785d07043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:08.324230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bece32-1bd6-46e9-83a9-316785d07043 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:08.324997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bece32-1bd6-46e9-83a9-316785d07043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:08.332010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bece32-1bd6-46e9-83a9-316785d07043 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:08.360364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=4,keystone:DELETE=11,keystone:SELECT=85 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:08.383394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 892/1783] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:08 2022] POST /identity/v3/users => generated 283 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:08.400488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9b6d8-9304-483a-9175-85354bd427a8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:08.400814 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9b6d8-9304-483a-9175-85354bd427a8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:08.401102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9b6d8-9304-483a-9175-85354bd427a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:08.429053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eae623-655c-4aea-b736-b056259a6981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:08.430024 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eae623-655c-4aea-b736-b056259a6981 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:08.430362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eae623-655c-4aea-b736-b056259a6981 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:08.430662 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eae623-655c-4aea-b736-b056259a6981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:08.433251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a0eae623-655c-4aea-b736-b056259a6981 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:08.441116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9ae9b6d8-9304-483a-9175-85354bd427a8 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:08.443293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9ae9b6d8-9304-483a-9175-85354bd427a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:08.467686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eae623-655c-4aea-b736-b056259a6981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:08.475752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eae623-655c-4aea-b736-b056259a6981 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:08.475862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 893/1784] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:08.475862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9b6d8-9304-483a-9175-85354bd427a8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:08.500146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 892/1785] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:08.503584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9b6d8-9304-483a-9175-85354bd427a8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:08.510549 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55a934-4bbf-438b-be61-6cbe6637fdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:08.511283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55a934-4bbf-438b-be61-6cbe6637fdd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:08.511565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55a934-4bbf-438b-be61-6cbe6637fdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:08.511938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55a934-4bbf-438b-be61-6cbe6637fdd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:08.512804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55a934-4bbf-438b-be61-6cbe6637fdd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:08.514608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55a934-4bbf-438b-be61-6cbe6637fdd9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:08.534782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 894/1786] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:08.540868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eae623-655c-4aea-b736-b056259a6981 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:08.553147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eae623-655c-4aea-b736-b056259a6981 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:08.560213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a3fcb-fce5-44af-ab5a-01125ecef120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:08.560941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a3fcb-fce5-44af-ab5a-01125ecef120 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:08.561231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a3fcb-fce5-44af-ab5a-01125ecef120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:08.561823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a3fcb-fce5-44af-ab5a-01125ecef120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:08.565054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a3fcb-fce5-44af-ab5a-01125ecef120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:08.567646 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55a934-4bbf-438b-be61-6cbe6637fdd9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:08.571280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1b55a934-4bbf-438b-be61-6cbe6637fdd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:08.574049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a3fcb-fce5-44af-ab5a-01125ecef120 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:08.589990 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 893/1787] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:04:08.596515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a3fcb-fce5-44af-ab5a-01125ecef120 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:08.617456 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a3fcb-fce5-44af-ab5a-01125ecef120 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:08.624123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55a934-4bbf-438b-be61-6cbe6637fdd9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:08.624892 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283bd43-ae19-4587-99d5-66ad789ffc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:08.626730 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283bd43-ae19-4587-99d5-66ad789ffc47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:08.627801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283bd43-ae19-4587-99d5-66ad789ffc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:08.629898 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283bd43-ae19-4587-99d5-66ad789ffc47 admin admin] PATH_INFO: `/v3/projects/4f565185e72d4317b9bdd7d2d4c7778d/users/817b662624924698ab7b18d7d264faa6/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:08.637076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfcfc5-0ae1-4262-8a57-65792d62b327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:08.638053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfcfc5-0ae1-4262-8a57-65792d62b327 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:08.638462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfcfc5-0ae1-4262-8a57-65792d62b327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:08.638787 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfcfc5-0ae1-4262-8a57-65792d62b327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:08.664808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283bd43-ae19-4587-99d5-66ad789ffc47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f565185e72d4317b9bdd7d2d4c7778d, user_id=817b662624924698ab7b18d7d264faa6, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:08.667731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283bd43-ae19-4587-99d5-66ad789ffc47 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:08.680759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cfcfc5-0ae1-4262-8a57-65792d62b327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:08.683032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cfcfc5-0ae1-4262-8a57-65792d62b327 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:08.699993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 894/1788] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:08 2022] PUT /identity/v3/projects/4f565185e72d4317b9bdd7d2d4c7778d/users/817b662624924698ab7b18d7d264faa6/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:08.714767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896a8cd-b8f4-45e6-9612-5b54518214cb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:08.715237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896a8cd-b8f4-45e6-9612-5b54518214cb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:08.715720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896a8cd-b8f4-45e6-9612-5b54518214cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:08.737639 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 895/1789] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:08.847050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e896a8cd-b8f4-45e6-9612-5b54518214cb None None] MFA Rules not processed for user `817b662624924698ab7b18d7d264faa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:08.849739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e896a8cd-b8f4-45e6-9612-5b54518214cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:09.104720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:09.143500 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 895/1790] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:09.144637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfcfc5-0ae1-4262-8a57-65792d62b327 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:09.170148 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfcfc5-0ae1-4262-8a57-65792d62b327 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:09.372760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadea215-731a-4052-a7cb-a03e89e4bbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:09.374804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadea215-731a-4052-a7cb-a03e89e4bbc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:09.375728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadea215-731a-4052-a7cb-a03e89e4bbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:09.376594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadea215-731a-4052-a7cb-a03e89e4bbc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:09.378867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadea215-731a-4052-a7cb-a03e89e4bbc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:09.386256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadea215-731a-4052-a7cb-a03e89e4bbc1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:09.532158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 896/1791] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:09 2022] POST /identity/v3/projects => generated 363 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:09.537305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896a8cd-b8f4-45e6-9612-5b54518214cb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:09.617976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896a8cd-b8f4-45e6-9612-5b54518214cb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:09.797324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadea215-731a-4052-a7cb-a03e89e4bbc1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:09.814833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fadea215-731a-4052-a7cb-a03e89e4bbc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:09.973496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf16dd-11f9-4c64-a0d1-be90e6d51227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:09.978985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf16dd-11f9-4c64-a0d1-be90e6d51227 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:09.987718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf16dd-11f9-4c64-a0d1-be90e6d51227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:09.988714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf16dd-11f9-4c64-a0d1-be90e6d51227 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:09.991503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf16dd-11f9-4c64-a0d1-be90e6d51227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:10.000902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf16dd-11f9-4c64-a0d1-be90e6d51227 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:10.118146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 896/1792] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:09 2022] POST /identity/v3/users => generated 288 bytes in 582 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:10.125115 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf16dd-11f9-4c64-a0d1-be90e6d51227 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:10.173953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf16dd-11f9-4c64-a0d1-be90e6d51227 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:10.199353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323736a-1b7a-427c-b006-65860f071256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:10.201024 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323736a-1b7a-427c-b006-65860f071256 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:10.203735 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323736a-1b7a-427c-b006-65860f071256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:10.204556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323736a-1b7a-427c-b006-65860f071256 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:10.206421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323736a-1b7a-427c-b006-65860f071256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:10.219812 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323736a-1b7a-427c-b006-65860f071256 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:10.312825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 897/1793] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:10.313087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadea215-731a-4052-a7cb-a03e89e4bbc1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:10.314943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323736a-1b7a-427c-b006-65860f071256 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:10.334203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6cb5a-87cf-459f-ade4-cd6b4b0e1ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:10.344893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6cb5a-87cf-459f-ade4-cd6b4b0e1ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:10.346918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6cb5a-87cf-459f-ade4-cd6b4b0e1ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:10.347559 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6cb5a-87cf-459f-ade4-cd6b4b0e1ca5 admin admin] PATH_INFO: `/v3/users/6f98c5fb5cec43f3b0de7b796bc5fa37` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:10.352309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6cb5a-87cf-459f-ade4-cd6b4b0e1ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f98c5fb5cec43f3b0de7b796bc5fa37)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:10.370491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6cb5a-87cf-459f-ade4-cd6b4b0e1ca5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:10.398052 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323736a-1b7a-427c-b006-65860f071256 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:10.421343 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ed234-b626-45df-b170-a0302cafac02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:10.424626 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ed234-b626-45df-b170-a0302cafac02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:10.425080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ed234-b626-45df-b170-a0302cafac02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:10.425595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ed234-b626-45df-b170-a0302cafac02 admin admin] PATH_INFO: `/v3/projects/c9a5c75a0f8e449fa90faf132320ac5a/users/a0e254343d2b4ea1a05b2df64c3340ad/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:10.498713 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ed234-b626-45df-b170-a0302cafac02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a5c75a0f8e449fa90faf132320ac5a, user_id=a0e254343d2b4ea1a05b2df64c3340ad, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:10.504615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ed234-b626-45df-b170-a0302cafac02 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:10.558246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 898/1794] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:10 2022] PUT /identity/v3/projects/c9a5c75a0f8e449fa90faf132320ac5a/users/a0e254343d2b4ea1a05b2df64c3340ad/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:10.563792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ed234-b626-45df-b170-a0302cafac02 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:10.594097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a7f6cb5a-87cf-459f-ade4-cd6b4b0e1ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f98c5fb5cec43f3b0de7b796bc5fa37'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:10.596239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a7f6cb5a-87cf-459f-ade4-cd6b4b0e1ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f98c5fb5cec43f3b0de7b796bc5fa37'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:10.598262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a7f6cb5a-87cf-459f-ade4-cd6b4b0e1ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f98c5fb5cec43f3b0de7b796bc5fa37'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:10.606108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:10.646691 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a7f6cb5a-87cf-459f-ade4-cd6b4b0e1ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f98c5fb5cec43f3b0de7b796bc5fa37'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:10.696697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a7f6cb5a-87cf-459f-ade4-cd6b4b0e1ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f98c5fb5cec43f3b0de7b796bc5fa37'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:10.731140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 897/1795] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:09 2022] DELETE /identity/v3/users/6f98c5fb5cec43f3b0de7b796bc5fa37 => generated 0 bytes in 936 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:10.740207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d134b82-cade-4298-b20a-55035b688f30 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:10.740511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d134b82-cade-4298-b20a-55035b688f30 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:10.740814 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d134b82-cade-4298-b20a-55035b688f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:10.790341 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9d134b82-cade-4298-b20a-55035b688f30 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:10.791662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9d134b82-cade-4298-b20a-55035b688f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:10.850814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ed234-b626-45df-b170-a0302cafac02 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:11.024341 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 898/1796] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:11.063163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:11.092171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d134b82-cade-4298-b20a-55035b688f30 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:11.096077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9d134b82-cade-4298-b20a-55035b688f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:11.126424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e765b49-2b34-4741-9ae0-4b7beef8106a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:11.129290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e765b49-2b34-4741-9ae0-4b7beef8106a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:11.131895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e765b49-2b34-4741-9ae0-4b7beef8106a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:11.132575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e765b49-2b34-4741-9ae0-4b7beef8106a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:11.133706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e765b49-2b34-4741-9ae0-4b7beef8106a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:11.140374 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e765b49-2b34-4741-9ae0-4b7beef8106a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:11.169369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 899/1797] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 607 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:11.173516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e765b49-2b34-4741-9ae0-4b7beef8106a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:11.175945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1e765b49-2b34-4741-9ae0-4b7beef8106a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:11.211233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e765b49-2b34-4741-9ae0-4b7beef8106a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:11.239951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb619a-193a-4951-b29e-eca4292b7681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:11.242918 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb619a-193a-4951-b29e-eca4292b7681 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:11.243454 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb619a-193a-4951-b29e-eca4292b7681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:11.243749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb619a-193a-4951-b29e-eca4292b7681 admin admin] PATH_INFO: `/v3/projects/c9a5c75a0f8e449fa90faf132320ac5a/users/a0e254343d2b4ea1a05b2df64c3340ad/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:11.251309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb619a-193a-4951-b29e-eca4292b7681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a5c75a0f8e449fa90faf132320ac5a, user_id=a0e254343d2b4ea1a05b2df64c3340ad, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:11.255556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb619a-193a-4951-b29e-eca4292b7681 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:11.285722 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 900/1798] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:11 2022] PUT /identity/v3/projects/c9a5c75a0f8e449fa90faf132320ac5a/users/a0e254343d2b4ea1a05b2df64c3340ad/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:11.287834 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ead8f-2ca4-469e-94db-aada71bcb1b8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:11.288369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ead8f-2ca4-469e-94db-aada71bcb1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:11.293154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ead8f-2ca4-469e-94db-aada71bcb1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:11.349138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d134b82-cade-4298-b20a-55035b688f30 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:11.368570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-447ead8f-2ca4-469e-94db-aada71bcb1b8 None None] MFA Rules not processed for user `a0e254343d2b4ea1a05b2df64c3340ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:11.371174 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-447ead8f-2ca4-469e-94db-aada71bcb1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:11.508747 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ea1c0-d9e9-4047-897e-3be8e0a6fb8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:11.509850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ea1c0-d9e9-4047-897e-3be8e0a6fb8c service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:11.510322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ea1c0-d9e9-4047-897e-3be8e0a6fb8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:11.510776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ea1c0-d9e9-4047-897e-3be8e0a6fb8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:11.549665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 901/1799] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:11 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:11.567789 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ead8f-2ca4-469e-94db-aada71bcb1b8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:11.569661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-447ead8f-2ca4-469e-94db-aada71bcb1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:11.619286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:11.705652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ea1c0-d9e9-4047-897e-3be8e0a6fb8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:11.708161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ea1c0-d9e9-4047-897e-3be8e0a6fb8c service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:11.742974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 899/1800] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:04:11.761551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a6cb9-25f6-4b54-b1c2-ccd51c6f747d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:11.764758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a6cb9-25f6-4b54-b1c2-ccd51c6f747d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:11.765312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a6cb9-25f6-4b54-b1c2-ccd51c6f747d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:11.767174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 900/1801] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 08:04:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:04:11.825547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a6cb9-25f6-4b54-b1c2-ccd51c6f747d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:11.833097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0b9a6cb9-25f6-4b54-b1c2-ccd51c6f747d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:11.855153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ead8f-2ca4-469e-94db-aada71bcb1b8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:11.878386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7927a-b9d8-48cc-9a26-27ec359b9b94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:11.879236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7927a-b9d8-48cc-9a26-27ec359b9b94 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:11.879524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7927a-b9d8-48cc-9a26-27ec359b9b94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:11.883387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7927a-b9d8-48cc-9a26-27ec359b9b94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:11.907770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7927a-b9d8-48cc-9a26-27ec359b9b94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:11.909355 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7927a-b9d8-48cc-9a26-27ec359b9b94 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:11.920919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a6cb9-25f6-4b54-b1c2-ccd51c6f747d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:11.945005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:11.946783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:11.947251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:11.947725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] PATH_INFO: `/v3/projects/f7f3b9d038144a2893afd2281e681b89` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:11.950496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 902/1802] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:11 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:04:11.952470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7f3b9d038144a2893afd2281e681b89)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:11.973610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:11.994519 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7927a-b9d8-48cc-9a26-27ec359b9b94 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:11.995101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7bd7927a-b9d8-48cc-9a26-27ec359b9b94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:12.054141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7927a-b9d8-48cc-9a26-27ec359b9b94 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:12.065445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:12.074388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:12.078925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:12.080737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621 admin admin] PATH_INFO: `/v3/projects/1d9903110fae40a9932b10cd8aba3a0c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:12.086880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9903110fae40a9932b10cd8aba3a0c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:12.098628 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:12.239816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9903110fae40a9932b10cd8aba3a0c', 'request_id': 'req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:12.243966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7f3b9d038144a2893afd2281e681b89', 'request_id': 'req-e69b7aec-3b2c-4e81-9d63-19867c23ed32'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:12.251014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9903110fae40a9932b10cd8aba3a0c', 'request_id': 'req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:12.255945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7f3b9d038144a2893afd2281e681b89', 'request_id': 'req-e69b7aec-3b2c-4e81-9d63-19867c23ed32'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:12.258496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:12.267833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9903110fae40a9932b10cd8aba3a0c', 'request_id': 'req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:12.279031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7f3b9d038144a2893afd2281e681b89', 'request_id': 'req-e69b7aec-3b2c-4e81-9d63-19867c23ed32'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:12.308716 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621 admin admin] The token cache is being invalidate because project 1d9903110fae40a9932b10cd8aba3a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:12.309266 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:12.311307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-956150a6-34e0-46d3-b9bd-ce6bd6a8b621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:12.315975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 903/1803] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:11 2022] DELETE /identity/v3/projects/1d9903110fae40a9932b10cd8aba3a0c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:12.319992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eee65a-3a6e-4d7e-8be7-7cce85d28176 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:12.320557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eee65a-3a6e-4d7e-8be7-7cce85d28176 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:12.321155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eee65a-3a6e-4d7e-8be7-7cce85d28176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:12.325484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] The token cache is being invalidate because project f7f3b9d038144a2893afd2281e681b89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:12.326096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:12.328569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:12.335256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 901/1804] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:11 2022] DELETE /identity/v3/projects/f7f3b9d038144a2893afd2281e681b89 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:12.340712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:12.343910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:12.407953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c0eee65a-3a6e-4d7e-8be7-7cce85d28176 None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:12.409660 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c0eee65a-3a6e-4d7e-8be7-7cce85d28176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:12.542251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b7aec-3b2c-4e81-9d63-19867c23ed32 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:12.578223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 904/1805] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 08:04:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:12.580279 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eee65a-3a6e-4d7e-8be7-7cce85d28176 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:12.581581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c0eee65a-3a6e-4d7e-8be7-7cce85d28176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:12.789234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43577796-ac16-4211-af1a-91bb382c72ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:12.790452 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43577796-ac16-4211-af1a-91bb382c72ee service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:12.791004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43577796-ac16-4211-af1a-91bb382c72ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:12.791374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43577796-ac16-4211-af1a-91bb382c72ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:12.797600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-43577796-ac16-4211-af1a-91bb382c72ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:12.801338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eee65a-3a6e-4d7e-8be7-7cce85d28176 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:12.810524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d0957-ab3e-446c-8e88-f8915cff1a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:12.811919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d0957-ab3e-446c-8e88-f8915cff1a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:12.814634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d0957-ab3e-446c-8e88-f8915cff1a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:12.815000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d0957-ab3e-446c-8e88-f8915cff1a28 admin admin] PATH_INFO: `/v3/users/013d0b65d54640fa99ff61bad3ea6173` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:12.818088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d0957-ab3e-446c-8e88-f8915cff1a28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=013d0b65d54640fa99ff61bad3ea6173)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:12.825320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d0957-ab3e-446c-8e88-f8915cff1a28 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:12.894672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:13.057418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6e0d0957-ab3e-446c-8e88-f8915cff1a28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '013d0b65d54640fa99ff61bad3ea6173'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:13.061946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43577796-ac16-4211-af1a-91bb382c72ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:13.068721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43577796-ac16-4211-af1a-91bb382c72ee service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:13.083283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6e0d0957-ab3e-446c-8e88-f8915cff1a28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '013d0b65d54640fa99ff61bad3ea6173'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:13.126985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6e0d0957-ab3e-446c-8e88-f8915cff1a28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '013d0b65d54640fa99ff61bad3ea6173'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:13.142234 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6e0d0957-ab3e-446c-8e88-f8915cff1a28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '013d0b65d54640fa99ff61bad3ea6173'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:13.165445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 902/1806] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 829 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:13.167929 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43577796-ac16-4211-af1a-91bb382c72ee service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:13.172746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-43577796-ac16-4211-af1a-91bb382c72ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:13.230777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43577796-ac16-4211-af1a-91bb382c72ee service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:13.246722 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6e0d0957-ab3e-446c-8e88-f8915cff1a28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '013d0b65d54640fa99ff61bad3ea6173'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:13.249121 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:13.251121 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:13.251872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:13.252917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] PATH_INFO: `/v3/users/a650b556283e467ba3aca5d8b04f9855` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:13.256011 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a650b556283e467ba3aca5d8b04f9855)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:13.265063 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:13.309957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 905/1807] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:12 2022] DELETE /identity/v3/users/013d0b65d54640fa99ff61bad3ea6173 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:13.317282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc4923-16d5-4fc7-acee-06125936bbe6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:13.326072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc4923-16d5-4fc7-acee-06125936bbe6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:13.326849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc4923-16d5-4fc7-acee-06125936bbe6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:13.343794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 906/1808] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 08:04:13 2022] GET /identity => generated 270 bytes in 33 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:04:13.415374 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42388ec4-653b-416e-bb88-d6aa8f11464d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:13.415843 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42388ec4-653b-416e-bb88-d6aa8f11464d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:13.416332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42388ec4-653b-416e-bb88-d6aa8f11464d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:13.440057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:13.467573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-42388ec4-653b-416e-bb88-d6aa8f11464d None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:13.472066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-42388ec4-653b-416e-bb88-d6aa8f11464d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:13.484089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a650b556283e467ba3aca5d8b04f9855'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:13.486479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a650b556283e467ba3aca5d8b04f9855'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:13.488995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a650b556283e467ba3aca5d8b04f9855'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:13.534664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a650b556283e467ba3aca5d8b04f9855'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:13.638731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a650b556283e467ba3aca5d8b04f9855'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:13.675801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 903/1809] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:13 2022] DELETE /identity/v3/users/a650b556283e467ba3aca5d8b04f9855 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:13.756258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 907/1810] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 08:04:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:14.002570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:14.005284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:14.178104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be35af0-2869-4aad-800e-feddfdb2d7bb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:14.200950 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:14.202031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:14.202608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:14.203093 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] PATH_INFO: `/v3/projects/c3c1c96638294aad8b0d2c80b7f08499` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:14.207672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c1c96638294aad8b0d2c80b7f08499)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:14.212221 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:14.289479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:14.302424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c1c96638294aad8b0d2c80b7f08499', 'request_id': 'req-85b58f22-7288-4482-8234-8c5e8c3ff547'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:14.320593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c1c96638294aad8b0d2c80b7f08499', 'request_id': 'req-85b58f22-7288-4482-8234-8c5e8c3ff547'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:14.332945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c1c96638294aad8b0d2c80b7f08499', 'request_id': 'req-85b58f22-7288-4482-8234-8c5e8c3ff547'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:14.359978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] The token cache is being invalidate because project c3c1c96638294aad8b0d2c80b7f08499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:14.362031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:14.364578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:14.371893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 904/1811] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:14 2022] DELETE /identity/v3/projects/c3c1c96638294aad8b0d2c80b7f08499 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:14.373585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42388ec4-653b-416e-bb88-d6aa8f11464d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:14.577093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42388ec4-653b-416e-bb88-d6aa8f11464d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:14.747273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:14.748973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:14.749597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:14.750212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] PATH_INFO: `/v3/users/0d94dad8396941ea9f18144575e303bc` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:14.753923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d94dad8396941ea9f18144575e303bc)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:14.758951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:14.974317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:14.992835 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d94dad8396941ea9f18144575e303bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:15.004877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:15.007754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:15.032436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d94dad8396941ea9f18144575e303bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:15.089002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d94dad8396941ea9f18144575e303bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:15.090962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d94dad8396941ea9f18144575e303bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:15.170156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d94dad8396941ea9f18144575e303bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:15.209600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 908/1812] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:14 2022] DELETE /identity/v3/users/0d94dad8396941ea9f18144575e303bc => generated 0 bytes in 837 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:15.223834 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b58f22-7288-4482-8234-8c5e8c3ff547 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:15.410799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:15.412679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:15.620116 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211fe6d-e7e4-4013-a50f-bc013cd662a3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:15.626263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1bccb-d6d3-46ef-9248-2fd665e5b475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:15.627590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1bccb-d6d3-46ef-9248-2fd665e5b475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:15.627830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1bccb-d6d3-46ef-9248-2fd665e5b475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:15.628331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1bccb-d6d3-46ef-9248-2fd665e5b475 admin admin] PATH_INFO: `/v3/projects/b25d24fcf6e84923bfec1ed06ad723db` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:15.630020 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1bccb-d6d3-46ef-9248-2fd665e5b475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b25d24fcf6e84923bfec1ed06ad723db)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:15.632564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1bccb-d6d3-46ef-9248-2fd665e5b475 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:15.635435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b615a2d-7e0c-4f6b-8e73-75daebaa83cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:15.636613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b615a2d-7e0c-4f6b-8e73-75daebaa83cf service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:15.637121 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b615a2d-7e0c-4f6b-8e73-75daebaa83cf service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:15.637642 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b615a2d-7e0c-4f6b-8e73-75daebaa83cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:15.656725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5b615a2d-7e0c-4f6b-8e73-75daebaa83cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:15.736043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b615a2d-7e0c-4f6b-8e73-75daebaa83cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:15.740126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b615a2d-7e0c-4f6b-8e73-75daebaa83cf service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:15.774840 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-82e1bccb-d6d3-46ef-9248-2fd665e5b475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b25d24fcf6e84923bfec1ed06ad723db', 'request_id': 'req-82e1bccb-d6d3-46ef-9248-2fd665e5b475'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:15.790394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-82e1bccb-d6d3-46ef-9248-2fd665e5b475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b25d24fcf6e84923bfec1ed06ad723db', 'request_id': 'req-82e1bccb-d6d3-46ef-9248-2fd665e5b475'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:15.811730 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-82e1bccb-d6d3-46ef-9248-2fd665e5b475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b25d24fcf6e84923bfec1ed06ad723db', 'request_id': 'req-82e1bccb-d6d3-46ef-9248-2fd665e5b475'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:15.812519 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 905/1813] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 808 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:15.815934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b615a2d-7e0c-4f6b-8e73-75daebaa83cf service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:15.832513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5b615a2d-7e0c-4f6b-8e73-75daebaa83cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:15.853070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-82e1bccb-d6d3-46ef-9248-2fd665e5b475 admin admin] The token cache is being invalidate because project b25d24fcf6e84923bfec1ed06ad723db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:15.853364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-82e1bccb-d6d3-46ef-9248-2fd665e5b475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:15.854748 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-82e1bccb-d6d3-46ef-9248-2fd665e5b475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:15.861425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 909/1814] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:15 2022] DELETE /identity/v3/projects/b25d24fcf6e84923bfec1ed06ad723db => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:15.869154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce2ece-07fd-4987-b5dc-a2841ac97868 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:15.869681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce2ece-07fd-4987-b5dc-a2841ac97868 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:15.870094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce2ece-07fd-4987-b5dc-a2841ac97868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:15.918710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b615a2d-7e0c-4f6b-8e73-75daebaa83cf service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:15.927698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dcce2ece-07fd-4987-b5dc-a2841ac97868 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:15.930900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dcce2ece-07fd-4987-b5dc-a2841ac97868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:16.077060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:16.131920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31d90b-e38e-4c98-a0c9-1dec5c11d02c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:16.132641 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31d90b-e38e-4c98-a0c9-1dec5c11d02c service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.132931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31d90b-e38e-4c98-a0c9-1dec5c11d02c service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.133211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31d90b-e38e-4c98-a0c9-1dec5c11d02c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.143730 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1e31d90b-e38e-4c98-a0c9-1dec5c11d02c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:16.174319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31d90b-e38e-4c98-a0c9-1dec5c11d02c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:16.176330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31d90b-e38e-4c98-a0c9-1dec5c11d02c service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:16.179149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:16.212079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 906/1815] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:16.216345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31d90b-e38e-4c98-a0c9-1dec5c11d02c service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:16.219314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1e31d90b-e38e-4c98-a0c9-1dec5c11d02c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:16.250588 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 910/1816] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:16.276781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6969f-fc91-43e9-a3f0-6eb8ae84d9cf None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.277515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6969f-fc91-43e9-a3f0-6eb8ae84d9cf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.278974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6969f-fc91-43e9-a3f0-6eb8ae84d9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.355818 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-44d6969f-fc91-43e9-a3f0-6eb8ae84d9cf None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:16.358350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-44d6969f-fc91-43e9-a3f0-6eb8ae84d9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:16.375227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 911/1817] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:16.391053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6969f-fc91-43e9-a3f0-6eb8ae84d9cf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:16.399309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31d90b-e38e-4c98-a0c9-1dec5c11d02c service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:16.404939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6969f-fc91-43e9-a3f0-6eb8ae84d9cf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:16.412363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068efab-b0dd-45fc-ba73-64c6f49ee7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:16.413344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068efab-b0dd-45fc-ba73-64c6f49ee7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.413792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068efab-b0dd-45fc-ba73-64c6f49ee7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.414322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068efab-b0dd-45fc-ba73-64c6f49ee7c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.415545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068efab-b0dd-45fc-ba73-64c6f49ee7c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:04:16.416059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068efab-b0dd-45fc-ba73-64c6f49ee7c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:16.418603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:16.418872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068efab-b0dd-45fc-ba73-64c6f49ee7c3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:16.419025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.419211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.419496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3 admin admin] PATH_INFO: `/v3/projects/228aad6cbc6241ebb3465c049d297c83` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.421460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=228aad6cbc6241ebb3465c049d297c83)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:16.431151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:16.440329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 912/1818] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:04:16.446717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068efab-b0dd-45fc-ba73-64c6f49ee7c3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:16.463149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068efab-b0dd-45fc-ba73-64c6f49ee7c3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:16.470954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d8e4f-40cb-4c39-955b-dc07bc99d467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:16.472837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d8e4f-40cb-4c39-955b-dc07bc99d467 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.472837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d8e4f-40cb-4c39-955b-dc07bc99d467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.473284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d8e4f-40cb-4c39-955b-dc07bc99d467 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.474286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d8e4f-40cb-4c39-955b-dc07bc99d467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:16.476899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d8e4f-40cb-4c39-955b-dc07bc99d467 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:16.552009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 913/1819] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:16 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:16.556032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d8e4f-40cb-4c39-955b-dc07bc99d467 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:16.598667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '228aad6cbc6241ebb3465c049d297c83', 'request_id': 'req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:16.634401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '228aad6cbc6241ebb3465c049d297c83', 'request_id': 'req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:16.649908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '228aad6cbc6241ebb3465c049d297c83', 'request_id': 'req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:16.673862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3 admin admin] The token cache is being invalidate because project 228aad6cbc6241ebb3465c049d297c83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:16.674225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:16.677137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4d16747a-93c4-4a74-9a3b-c71e6bb641b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:16.679307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 907/1820] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:16 2022] DELETE /identity/v3/projects/228aad6cbc6241ebb3465c049d297c83 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:16.686353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b385145-9e76-4653-985a-13b3047eee46 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.686677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b385145-9e76-4653-985a-13b3047eee46 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.686922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b385145-9e76-4653-985a-13b3047eee46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.737111 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4b385145-9e76-4653-985a-13b3047eee46 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:16.738904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4b385145-9e76-4653-985a-13b3047eee46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:16.755057 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 908/1821] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:16.755304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d8e4f-40cb-4c39-955b-dc07bc99d467 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:16.760867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f91b3-59d6-4b0c-8085-ae5dae79fbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:16.761833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f91b3-59d6-4b0c-8085-ae5dae79fbd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.762188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f91b3-59d6-4b0c-8085-ae5dae79fbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.764203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625d15a-66dd-46f7-964e-f7c72925f357 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.764596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625d15a-66dd-46f7-964e-f7c72925f357 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.764971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625d15a-66dd-46f7-964e-f7c72925f357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.766733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f91b3-59d6-4b0c-8085-ae5dae79fbd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.769583 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f91b3-59d6-4b0c-8085-ae5dae79fbd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:16.774545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f91b3-59d6-4b0c-8085-ae5dae79fbd6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:16.802091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9625d15a-66dd-46f7-964e-f7c72925f357 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:16.803645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9625d15a-66dd-46f7-964e-f7c72925f357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:16.820197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 914/1822] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:16 2022] POST /identity/v3/users => generated 285 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:16.825265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f91b3-59d6-4b0c-8085-ae5dae79fbd6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:16.833668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 909/1823] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:16.835495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d92f91b3-59d6-4b0c-8085-ae5dae79fbd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:16.837501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625d15a-66dd-46f7-964e-f7c72925f357 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:16.858040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f91b3-59d6-4b0c-8085-ae5dae79fbd6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:16.864190 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c69a9-6f65-4598-b27c-c649aa8b753a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:16.865010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c69a9-6f65-4598-b27c-c649aa8b753a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.865750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c69a9-6f65-4598-b27c-c649aa8b753a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.866049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c69a9-6f65-4598-b27c-c649aa8b753a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.867093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c69a9-6f65-4598-b27c-c649aa8b753a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:16.868147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625d15a-66dd-46f7-964e-f7c72925f357 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:16.869111 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c69a9-6f65-4598-b27c-c649aa8b753a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:16.889358 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 915/1824] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:16.890970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c69a9-6f65-4598-b27c-c649aa8b753a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:16.893099 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc273e7-df87-400e-baf4-7011f50326db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:16.894352 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc273e7-df87-400e-baf4-7011f50326db admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.894859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc273e7-df87-400e-baf4-7011f50326db admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.895604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc273e7-df87-400e-baf4-7011f50326db admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.897068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc273e7-df87-400e-baf4-7011f50326db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:04:16.897672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc273e7-df87-400e-baf4-7011f50326db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:16.900888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc273e7-df87-400e-baf4-7011f50326db admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:16.909211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c69a9-6f65-4598-b27c-c649aa8b753a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:16.916541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7644c-2566-4a31-803b-c70f3d951fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:16.917555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7644c-2566-4a31-803b-c70f3d951fee admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.918651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7644c-2566-4a31-803b-c70f3d951fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.919085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7644c-2566-4a31-803b-c70f3d951fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.920013 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7644c-2566-4a31-803b-c70f3d951fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:16.922821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7644c-2566-4a31-803b-c70f3d951fee admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:16.944308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 916/1825] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:16.945363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7644c-2566-4a31-803b-c70f3d951fee admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:16.951320 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 910/1826] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:04:16.952977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc273e7-df87-400e-baf4-7011f50326db admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:16.958677 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7644c-2566-4a31-803b-c70f3d951fee admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:16.965993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a750954-4746-4e15-a175-19e06c7b46a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:16.967022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a750954-4746-4e15-a175-19e06c7b46a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.967370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a750954-4746-4e15-a175-19e06c7b46a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.967655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a750954-4746-4e15-a175-19e06c7b46a2 admin admin] PATH_INFO: `/v3/projects/b8d567be1b3a43d18a9f185416845815/users/87d8f1a5a0324ca893ddaab1905023d4/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.967874 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc273e7-df87-400e-baf4-7011f50326db admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:16.975186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c1f20-5a15-4a91-aa31-3ae15de376b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:16.976060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c1f20-5a15-4a91-aa31-3ae15de376b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:16.976490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c1f20-5a15-4a91-aa31-3ae15de376b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:16.976911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c1f20-5a15-4a91-aa31-3ae15de376b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:16.978008 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c1f20-5a15-4a91-aa31-3ae15de376b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:16.980171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c1f20-5a15-4a91-aa31-3ae15de376b5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:16.994994 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a750954-4746-4e15-a175-19e06c7b46a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d567be1b3a43d18a9f185416845815, user_id=87d8f1a5a0324ca893ddaab1905023d4, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:16.997941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a750954-4746-4e15-a175-19e06c7b46a2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:17.018469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 917/1827] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:16 2022] PUT /identity/v3/projects/b8d567be1b3a43d18a9f185416845815/users/87d8f1a5a0324ca893ddaab1905023d4/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:17.033107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62752b8-b739-4de8-b295-5b9fdeb862db None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:17.033107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62752b8-b739-4de8-b295-5b9fdeb862db None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:17.033107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62752b8-b739-4de8-b295-5b9fdeb862db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:17.034049 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 911/1828] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:16 2022] POST /identity/v3/projects => generated 387 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:17.034989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c1f20-5a15-4a91-aa31-3ae15de376b5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:17.068604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a62752b8-b739-4de8-b295-5b9fdeb862db None None] MFA Rules not processed for user `87d8f1a5a0324ca893ddaab1905023d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:17.070363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a62752b8-b739-4de8-b295-5b9fdeb862db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:17.073469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:17.128745 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:17.146138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c1f20-5a15-4a91-aa31-3ae15de376b5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:17.152608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab56a78-263f-4093-be1d-6bace88c1233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:17.153372 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab56a78-263f-4093-be1d-6bace88c1233 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:17.153683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab56a78-263f-4093-be1d-6bace88c1233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:17.154079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab56a78-263f-4093-be1d-6bace88c1233 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:17.154872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab56a78-263f-4093-be1d-6bace88c1233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:17.157044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab56a78-263f-4093-be1d-6bace88c1233 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:17.167790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 918/1829] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:17.183827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62752b8-b739-4de8-b295-5b9fdeb862db None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:17.186422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a62752b8-b739-4de8-b295-5b9fdeb862db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:17.210357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 912/1830] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:17 2022] POST /identity/v3/users => generated 294 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:17.215175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab56a78-263f-4093-be1d-6bace88c1233 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:17.231390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab56a78-263f-4093-be1d-6bace88c1233 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:17.239542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9815069-e626-40ab-a038-9ebd5b61b2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:17.241143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9815069-e626-40ab-a038-9ebd5b61b2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:17.241143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9815069-e626-40ab-a038-9ebd5b61b2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:17.241342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9815069-e626-40ab-a038-9ebd5b61b2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:17.242100 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9815069-e626-40ab-a038-9ebd5b61b2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:17.244140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9815069-e626-40ab-a038-9ebd5b61b2c3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:17.263325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 913/1831] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:04:17.264437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9815069-e626-40ab-a038-9ebd5b61b2c3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:17.276722 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9815069-e626-40ab-a038-9ebd5b61b2c3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:17.283672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aceaec6-b41d-438b-a7d9-89c749dde1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:17.284506 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aceaec6-b41d-438b-a7d9-89c749dde1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:17.285046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aceaec6-b41d-438b-a7d9-89c749dde1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:17.285566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aceaec6-b41d-438b-a7d9-89c749dde1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:17.286548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aceaec6-b41d-438b-a7d9-89c749dde1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:17.305662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aceaec6-b41d-438b-a7d9-89c749dde1c1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:17.333890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 914/1832] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:17.335962 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aceaec6-b41d-438b-a7d9-89c749dde1c1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:17.352893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aceaec6-b41d-438b-a7d9-89c749dde1c1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:17.362077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85441e59-c745-412a-bce5-2b0a5ccd8f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:17.363043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62752b8-b739-4de8-b295-5b9fdeb862db None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:17.363393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85441e59-c745-412a-bce5-2b0a5ccd8f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:17.363914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85441e59-c745-412a-bce5-2b0a5ccd8f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:17.364207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85441e59-c745-412a-bce5-2b0a5ccd8f62 admin admin] PATH_INFO: `/v3/projects/0e910a45f6c640ffb173f1fcbbb8e95f/users/54088a435c794d399ea3c0422e2d113b/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:17.369451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a52f7-d9f2-4330-a5f1-1532095d8811 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a122422a45446c9a63d67b4f931fa47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:17.370398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a52f7-d9f2-4330-a5f1-1532095d8811 service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:17.370786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a52f7-d9f2-4330-a5f1-1532095d8811 service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:17.371259 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a52f7-d9f2-4330-a5f1-1532095d8811 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:17.383086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a52f7-d9f2-4330-a5f1-1532095d8811 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:17.385506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a52f7-d9f2-4330-a5f1-1532095d8811 service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:17.389470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85441e59-c745-412a-bce5-2b0a5ccd8f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e910a45f6c640ffb173f1fcbbb8e95f, user_id=54088a435c794d399ea3c0422e2d113b, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:17.393577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85441e59-c745-412a-bce5-2b0a5ccd8f62 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:17.398113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 919/1833] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:17 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:17.425245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 915/1834] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:17 2022] PUT /identity/v3/projects/0e910a45f6c640ffb173f1fcbbb8e95f/users/54088a435c794d399ea3c0422e2d113b/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:17.426631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faca76-c914-4dbb-a2f0-21269e455af5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:17.426631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faca76-c914-4dbb-a2f0-21269e455af5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:17.426903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faca76-c914-4dbb-a2f0-21269e455af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:17.464458 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-63faca76-c914-4dbb-a2f0-21269e455af5 None None] MFA Rules not processed for user `54088a435c794d399ea3c0422e2d113b`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:17.465985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-63faca76-c914-4dbb-a2f0-21269e455af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:17.587213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 920/1835] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:17.592418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85441e59-c745-412a-bce5-2b0a5ccd8f62 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:17.740847 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85441e59-c745-412a-bce5-2b0a5ccd8f62 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:17.747443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6fdf6-e16b-4f3c-8f92-c4b5083f741e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:17.748464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6fdf6-e16b-4f3c-8f92-c4b5083f741e admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:17.748878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6fdf6-e16b-4f3c-8f92-c4b5083f741e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:17.749213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6fdf6-e16b-4f3c-8f92-c4b5083f741e admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:17.750187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6fdf6-e16b-4f3c-8f92-c4b5083f741e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:17.752909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6fdf6-e16b-4f3c-8f92-c4b5083f741e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:17.826431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 916/1836] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:17 2022] POST /identity/v3/projects => generated 387 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:17.828342 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faca76-c914-4dbb-a2f0-21269e455af5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:17.879319 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faca76-c914-4dbb-a2f0-21269e455af5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:18.075034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90032680-4f56-4d7e-be1e-aa15d721f494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:18.076257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90032680-4f56-4d7e-be1e-aa15d721f494 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:18.076764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90032680-4f56-4d7e-be1e-aa15d721f494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:18.077226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90032680-4f56-4d7e-be1e-aa15d721f494 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:18.078958 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90032680-4f56-4d7e-be1e-aa15d721f494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:18.082992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90032680-4f56-4d7e-be1e-aa15d721f494 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:18.132093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 921/1837] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:17 2022] POST /identity/v3/users => generated 300 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:18.137369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6fdf6-e16b-4f3c-8f92-c4b5083f741e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:18.169650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6fdf6-e16b-4f3c-8f92-c4b5083f741e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:18.176376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bab2e-7bb0-4b54-bb89-24a96eacdf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:18.177146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bab2e-7bb0-4b54-bb89-24a96eacdf87 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:18.178696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bab2e-7bb0-4b54-bb89-24a96eacdf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:18.178696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bab2e-7bb0-4b54-bb89-24a96eacdf87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:18.178993 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bab2e-7bb0-4b54-bb89-24a96eacdf87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:18.181380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bab2e-7bb0-4b54-bb89-24a96eacdf87 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:18.207685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 917/1838] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:18.211058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90032680-4f56-4d7e-be1e-aa15d721f494 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:18.225604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:18.249964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90032680-4f56-4d7e-be1e-aa15d721f494 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:18.279450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46182069-5899-4dd6-a2cb-e9f3765b151b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:18.283209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46182069-5899-4dd6-a2cb-e9f3765b151b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:18.285768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46182069-5899-4dd6-a2cb-e9f3765b151b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:18.289381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46182069-5899-4dd6-a2cb-e9f3765b151b admin admin] PATH_INFO: `/v3/projects/bf36e39a21d6497ea08c85e08fc9ceab/users/e74556db47b44db1bf4086f4711dc50c/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:18.398624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46182069-5899-4dd6-a2cb-e9f3765b151b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf36e39a21d6497ea08c85e08fc9ceab, user_id=e74556db47b44db1bf4086f4711dc50c, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:18.407712 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46182069-5899-4dd6-a2cb-e9f3765b151b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:18.441105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 922/1839] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:18 2022] PUT /identity/v3/projects/bf36e39a21d6497ea08c85e08fc9ceab/users/e74556db47b44db1bf4086f4711dc50c/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:18.444922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bab2e-7bb0-4b54-bb89-24a96eacdf87 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:18.482598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bab2e-7bb0-4b54-bb89-24a96eacdf87 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:18.616029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:18.666731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86786904-8890-4841-ba3b-4a19adbd54f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:18.667626 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86786904-8890-4841-ba3b-4a19adbd54f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:18.668033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86786904-8890-4841-ba3b-4a19adbd54f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:18.668531 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86786904-8890-4841-ba3b-4a19adbd54f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:18.670035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86786904-8890-4841-ba3b-4a19adbd54f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:18.673675 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86786904-8890-4841-ba3b-4a19adbd54f6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:18.711584 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46182069-5899-4dd6-a2cb-e9f3765b151b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:18.732853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 918/1840] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:04:18.738321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86786904-8890-4841-ba3b-4a19adbd54f6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:18.743627 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-86786904-8890-4841-ba3b-4a19adbd54f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:18.755283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46182069-5899-4dd6-a2cb-e9f3765b151b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:18.792698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a261d-c09f-47b7-8842-25943a291751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:18.794239 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a261d-c09f-47b7-8842-25943a291751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:18.794709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a261d-c09f-47b7-8842-25943a291751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:18.795163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a261d-c09f-47b7-8842-25943a291751 admin admin] PATH_INFO: `/v3/projects/bf36e39a21d6497ea08c85e08fc9ceab/users/e74556db47b44db1bf4086f4711dc50c/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:18.810017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a261d-c09f-47b7-8842-25943a291751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf36e39a21d6497ea08c85e08fc9ceab, user_id=e74556db47b44db1bf4086f4711dc50c, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:18.812070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86786904-8890-4841-ba3b-4a19adbd54f6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:18.816329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a261d-c09f-47b7-8842-25943a291751 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:18.833043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed656d-10bd-4583-b2bb-7f0439757dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:18.836665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed656d-10bd-4583-b2bb-7f0439757dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:18.837235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed656d-10bd-4583-b2bb-7f0439757dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:18.837708 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed656d-10bd-4583-b2bb-7f0439757dc9 admin admin] PATH_INFO: `/v3/projects/45d5f7dae38b4583930d78d695304eb4` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:18.847159 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed656d-10bd-4583-b2bb-7f0439757dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d5f7dae38b4583930d78d695304eb4)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:18.849127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed656d-10bd-4583-b2bb-7f0439757dc9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:18.851405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 923/1841] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:18 2022] PUT /identity/v3/projects/bf36e39a21d6497ea08c85e08fc9ceab/users/e74556db47b44db1bf4086f4711dc50c/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:18.864388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842111c-34c7-4752-af62-095060d43058 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:18.865173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842111c-34c7-4752-af62-095060d43058 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:18.865697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842111c-34c7-4752-af62-095060d43058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:18.929057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9842111c-34c7-4752-af62-095060d43058 None None] MFA Rules not processed for user `e74556db47b44db1bf4086f4711dc50c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:18.931036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9842111c-34c7-4752-af62-095060d43058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:18.933562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cbed656d-10bd-4583-b2bb-7f0439757dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d5f7dae38b4583930d78d695304eb4', 'request_id': 'req-cbed656d-10bd-4583-b2bb-7f0439757dc9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:18.942895 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cbed656d-10bd-4583-b2bb-7f0439757dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d5f7dae38b4583930d78d695304eb4', 'request_id': 'req-cbed656d-10bd-4583-b2bb-7f0439757dc9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:18.953560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cbed656d-10bd-4583-b2bb-7f0439757dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d5f7dae38b4583930d78d695304eb4', 'request_id': 'req-cbed656d-10bd-4583-b2bb-7f0439757dc9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:18.975420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cbed656d-10bd-4583-b2bb-7f0439757dc9 admin admin] The token cache is being invalidate because project 45d5f7dae38b4583930d78d695304eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:18.975712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cbed656d-10bd-4583-b2bb-7f0439757dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:18.976408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cbed656d-10bd-4583-b2bb-7f0439757dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:18.978096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 919/1842] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:18 2022] DELETE /identity/v3/projects/45d5f7dae38b4583930d78d695304eb4 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:18.985335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50df97-2bec-4f89-a67e-e71ba8c87205 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:18.986257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50df97-2bec-4f89-a67e-e71ba8c87205 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:18.986759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50df97-2bec-4f89-a67e-e71ba8c87205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:19.014146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4e50df97-2bec-4f89-a67e-e71ba8c87205 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:19.015548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4e50df97-2bec-4f89-a67e-e71ba8c87205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:19.058233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 924/1843] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:19.079493 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9842111c-34c7-4752-af62-095060d43058 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:19.081495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9842111c-34c7-4752-af62-095060d43058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:19.181164 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9842111c-34c7-4752-af62-095060d43058 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:19.186102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324073e-c395-4a25-a138-25027ae876ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:19.186867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324073e-c395-4a25-a138-25027ae876ba service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:19.187156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324073e-c395-4a25-a138-25027ae876ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:19.187436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324073e-c395-4a25-a138-25027ae876ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:19.199786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324073e-c395-4a25-a138-25027ae876ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:19.201515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324073e-c395-4a25-a138-25027ae876ba service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:19.215236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 925/1844] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:04:19.222070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 920/1845] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:19.236833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911546a-ebb6-4961-a4a5-b077929885a7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:19.237509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911546a-ebb6-4961-a4a5-b077929885a7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:19.238174 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911546a-ebb6-4961-a4a5-b077929885a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:19.275691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a911546a-ebb6-4961-a4a5-b077929885a7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:19.278011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a911546a-ebb6-4961-a4a5-b077929885a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:19.293025 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 926/1846] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:19.298061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50df97-2bec-4f89-a67e-e71ba8c87205 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:19.329223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50df97-2bec-4f89-a67e-e71ba8c87205 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:19.343310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653be18-8f55-4592-88f0-bfcea507442c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:19.344584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653be18-8f55-4592-88f0-bfcea507442c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:19.344968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653be18-8f55-4592-88f0-bfcea507442c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:19.345910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653be18-8f55-4592-88f0-bfcea507442c admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:19.346541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653be18-8f55-4592-88f0-bfcea507442c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:04:19.347019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653be18-8f55-4592-88f0-bfcea507442c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:19.350180 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653be18-8f55-4592-88f0-bfcea507442c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:19.381689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 921/1847] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:04:19.389564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911546a-ebb6-4961-a4a5-b077929885a7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:19.400559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911546a-ebb6-4961-a4a5-b077929885a7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:19.407418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d51cd2-c24e-478c-8a93-c6842c130359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:19.408390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d51cd2-c24e-478c-8a93-c6842c130359 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:19.408737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d51cd2-c24e-478c-8a93-c6842c130359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:19.409215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d51cd2-c24e-478c-8a93-c6842c130359 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:19.410058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d51cd2-c24e-478c-8a93-c6842c130359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:19.412446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d51cd2-c24e-478c-8a93-c6842c130359 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:19.452995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 927/1848] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:19 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:04:19.457122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653be18-8f55-4592-88f0-bfcea507442c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:19.469398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653be18-8f55-4592-88f0-bfcea507442c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:19.521050 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:19.542898 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54045e-9264-45a6-8042-4916e7819dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:19.543969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54045e-9264-45a6-8042-4916e7819dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:19.544417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54045e-9264-45a6-8042-4916e7819dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:19.544859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54045e-9264-45a6-8042-4916e7819dd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:19.546094 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54045e-9264-45a6-8042-4916e7819dd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:19.549556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54045e-9264-45a6-8042-4916e7819dd1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:19.585845 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 922/1849] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:19 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:04:19.587833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d51cd2-c24e-478c-8a93-c6842c130359 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:19.612187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d51cd2-c24e-478c-8a93-c6842c130359 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:19.620308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8e197-3352-46d4-925a-d121e581097e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:19.621053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8e197-3352-46d4-925a-d121e581097e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:19.621680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8e197-3352-46d4-925a-d121e581097e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:19.621854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8e197-3352-46d4-925a-d121e581097e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:19.623315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8e197-3352-46d4-925a-d121e581097e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:19.625093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8e197-3352-46d4-925a-d121e581097e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:19.644356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 928/1850] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:19.649121 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54045e-9264-45a6-8042-4916e7819dd1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:19.668069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54045e-9264-45a6-8042-4916e7819dd1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:19.677197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8fd10-4374-49aa-a18b-9ccd71582bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:19.677197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8fd10-4374-49aa-a18b-9ccd71582bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:19.677197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8fd10-4374-49aa-a18b-9ccd71582bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:19.677197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8fd10-4374-49aa-a18b-9ccd71582bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:19.677591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8fd10-4374-49aa-a18b-9ccd71582bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:19.677717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8fd10-4374-49aa-a18b-9ccd71582bb8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:19.690762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 923/1851] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:19.695115 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8e197-3352-46d4-925a-d121e581097e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:19.706120 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8e197-3352-46d4-925a-d121e581097e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:19.712527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d119ff4-64dc-4a3a-b2b2-edfdbe8a4cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:19.713956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d119ff4-64dc-4a3a-b2b2-edfdbe8a4cf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:19.714291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d119ff4-64dc-4a3a-b2b2-edfdbe8a4cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:19.714627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d119ff4-64dc-4a3a-b2b2-edfdbe8a4cf0 admin admin] PATH_INFO: `/v3/projects/1ef6a9c1c98146e181aefee41790b4ff/users/23c1c7866d8b44e1a4700bde0f9a6839/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:19.731969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:19.737021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d119ff4-64dc-4a3a-b2b2-edfdbe8a4cf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef6a9c1c98146e181aefee41790b4ff, user_id=23c1c7866d8b44e1a4700bde0f9a6839, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:19.741248 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d119ff4-64dc-4a3a-b2b2-edfdbe8a4cf0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:19.761065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 929/1852] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:19 2022] PUT /identity/v3/projects/1ef6a9c1c98146e181aefee41790b4ff/users/23c1c7866d8b44e1a4700bde0f9a6839/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:19.774653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec2b09-537b-4a8f-80ba-36722133e8b8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:19.774915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec2b09-537b-4a8f-80ba-36722133e8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:19.775213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec2b09-537b-4a8f-80ba-36722133e8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:19.812761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-94ec2b09-537b-4a8f-80ba-36722133e8b8 None None] MFA Rules not processed for user `23c1c7866d8b44e1a4700bde0f9a6839`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:19.814137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-94ec2b09-537b-4a8f-80ba-36722133e8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:19.934324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 924/1853] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:19.973565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d119ff4-64dc-4a3a-b2b2-edfdbe8a4cf0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:19.980415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6d119ff4-64dc-4a3a-b2b2-edfdbe8a4cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:20.058191 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d119ff4-64dc-4a3a-b2b2-edfdbe8a4cf0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:20.062137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556d052-064e-40e1-88a5-c413db323735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:20.062836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556d052-064e-40e1-88a5-c413db323735 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:20.063112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556d052-064e-40e1-88a5-c413db323735 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:20.063391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556d052-064e-40e1-88a5-c413db323735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:20.076075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556d052-064e-40e1-88a5-c413db323735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:20.077566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556d052-064e-40e1-88a5-c413db323735 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:20.088345 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 930/1854] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:20.092374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec2b09-537b-4a8f-80ba-36722133e8b8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:20.093991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-94ec2b09-537b-4a8f-80ba-36722133e8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:20.108955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556d052-064e-40e1-88a5-c413db323735 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:20.226785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556d052-064e-40e1-88a5-c413db323735 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:20.232543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ace90e-583f-4abe-96b7-7b28ab98cd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:20.233755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ace90e-583f-4abe-96b7-7b28ab98cd22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:20.234250 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ace90e-583f-4abe-96b7-7b28ab98cd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:20.234663 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ace90e-583f-4abe-96b7-7b28ab98cd22 admin admin] PATH_INFO: `/v3/users/23c1c7866d8b44e1a4700bde0f9a6839` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:20.236925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ace90e-583f-4abe-96b7-7b28ab98cd22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c1c7866d8b44e1a4700bde0f9a6839)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:20.239510 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ace90e-583f-4abe-96b7-7b28ab98cd22 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:20.250762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec2b09-537b-4a8f-80ba-36722133e8b8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:20.259368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f5633-61df-43de-b25e-60783194ef69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:20.260936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f5633-61df-43de-b25e-60783194ef69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:20.261543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f5633-61df-43de-b25e-60783194ef69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:20.262313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f5633-61df-43de-b25e-60783194ef69 admin admin] PATH_INFO: `/v3/users/54088a435c794d399ea3c0422e2d113b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:20.265203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f5633-61df-43de-b25e-60783194ef69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54088a435c794d399ea3c0422e2d113b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:20.269266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f5633-61df-43de-b25e-60783194ef69 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:20.348459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-15ace90e-583f-4abe-96b7-7b28ab98cd22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c1c7866d8b44e1a4700bde0f9a6839'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:20.364402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-15ace90e-583f-4abe-96b7-7b28ab98cd22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c1c7866d8b44e1a4700bde0f9a6839'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:20.376998 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-15ace90e-583f-4abe-96b7-7b28ab98cd22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c1c7866d8b44e1a4700bde0f9a6839'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:20.378873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-15ace90e-583f-4abe-96b7-7b28ab98cd22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c1c7866d8b44e1a4700bde0f9a6839'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:20.386599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-559f5633-61df-43de-b25e-60783194ef69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54088a435c794d399ea3c0422e2d113b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:20.388887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-559f5633-61df-43de-b25e-60783194ef69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54088a435c794d399ea3c0422e2d113b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:20.390008 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-559f5633-61df-43de-b25e-60783194ef69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54088a435c794d399ea3c0422e2d113b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:20.402553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-15ace90e-583f-4abe-96b7-7b28ab98cd22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c1c7866d8b44e1a4700bde0f9a6839'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:20.404879 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:20.408019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-559f5633-61df-43de-b25e-60783194ef69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54088a435c794d399ea3c0422e2d113b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:20.411882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 931/1855] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:20 2022] DELETE /identity/v3/users/23c1c7866d8b44e1a4700bde0f9a6839 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:20.417387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d930b-2cfb-45fe-b1af-e94717fae54a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:20.417814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d930b-2cfb-45fe-b1af-e94717fae54a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:20.418240 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d930b-2cfb-45fe-b1af-e94717fae54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:20.436900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-559f5633-61df-43de-b25e-60783194ef69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54088a435c794d399ea3c0422e2d113b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:20.447837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7f5d930b-2cfb-45fe-b1af-e94717fae54a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:20.449417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7f5d930b-2cfb-45fe-b1af-e94717fae54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:20.463389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:20.465679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 925/1856] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:20 2022] DELETE /identity/v3/users/54088a435c794d399ea3c0422e2d113b => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:20.472718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a5d05-ee95-4a07-972c-0dcf76f65e18 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:20.473547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a5d05-ee95-4a07-972c-0dcf76f65e18 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:20.474378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a5d05-ee95-4a07-972c-0dcf76f65e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:20.508350 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2e5a5d05-ee95-4a07-972c-0dcf76f65e18 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:20.510680 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2e5a5d05-ee95-4a07-972c-0dcf76f65e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:20.563308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 932/1857] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:20.579929 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d930b-2cfb-45fe-b1af-e94717fae54a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:20.582892 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7f5d930b-2cfb-45fe-b1af-e94717fae54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:20.624423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 926/1858] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:20.636910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a5d05-ee95-4a07-972c-0dcf76f65e18 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:20.640522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2e5a5d05-ee95-4a07-972c-0dcf76f65e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:20.686610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d930b-2cfb-45fe-b1af-e94717fae54a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:20.722977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a5d05-ee95-4a07-972c-0dcf76f65e18 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:20.728921 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b70725-abd3-46dd-ac3a-b51ac14a6aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:20.730274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b70725-abd3-46dd-ac3a-b51ac14a6aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:20.730564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b70725-abd3-46dd-ac3a-b51ac14a6aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:20.730853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b70725-abd3-46dd-ac3a-b51ac14a6aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:20.745222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b70725-abd3-46dd-ac3a-b51ac14a6aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:20.750545 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b70725-abd3-46dd-ac3a-b51ac14a6aa0 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:20.764763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 927/1859] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:04:20.930949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299eaa9f-a9b0-4a12-ad79-742c006d9db0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:20.931636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299eaa9f-a9b0-4a12-ad79-742c006d9db0 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:20.931917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299eaa9f-a9b0-4a12-ad79-742c006d9db0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:20.932196 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299eaa9f-a9b0-4a12-ad79-742c006d9db0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:20.946362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299eaa9f-a9b0-4a12-ad79-742c006d9db0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:20.949602 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299eaa9f-a9b0-4a12-ad79-742c006d9db0 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:20.968389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 933/1860] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:21.441994 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b70725-abd3-46dd-ac3a-b51ac14a6aa0 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:21.444783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-79b70725-abd3-46dd-ac3a-b51ac14a6aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:21.463498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b70725-abd3-46dd-ac3a-b51ac14a6aa0 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:21.470436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:21.471556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:21.472913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:21.473423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] PATH_INFO: `/v3/projects/0e910a45f6c640ffb173f1fcbbb8e95f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:21.475320 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e910a45f6c640ffb173f1fcbbb8e95f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:21.478665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:21.568455 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e910a45f6c640ffb173f1fcbbb8e95f', 'request_id': 'req-c8535ddc-7a66-4e71-83ed-606b42508544'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:21.582007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e910a45f6c640ffb173f1fcbbb8e95f', 'request_id': 'req-c8535ddc-7a66-4e71-83ed-606b42508544'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:21.596872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e910a45f6c640ffb173f1fcbbb8e95f', 'request_id': 'req-c8535ddc-7a66-4e71-83ed-606b42508544'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:21.612189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:21.623937 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] The token cache is being invalidate because project 0e910a45f6c640ffb173f1fcbbb8e95f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:21.624465 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:21.627694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:21.654176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 928/1861] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:21 2022] DELETE /identity/v3/projects/0e910a45f6c640ffb173f1fcbbb8e95f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:21.655641 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299eaa9f-a9b0-4a12-ad79-742c006d9db0 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:21.677474 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:21.685703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299eaa9f-a9b0-4a12-ad79-742c006d9db0 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:21.692832 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:21.932761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8535ddc-7a66-4e71-83ed-606b42508544 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:21.945648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670feec-cc2c-4481-8788-9005013f3623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:21.946804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670feec-cc2c-4481-8788-9005013f3623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:21.947260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670feec-cc2c-4481-8788-9005013f3623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:21.947663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670feec-cc2c-4481-8788-9005013f3623 admin admin] PATH_INFO: `/v3/projects/1ef6a9c1c98146e181aefee41790b4ff` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:21.949601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670feec-cc2c-4481-8788-9005013f3623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ef6a9c1c98146e181aefee41790b4ff)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:21.953418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670feec-cc2c-4481-8788-9005013f3623 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:21.977522 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5b085-5cbd-4b2e-97a3-6f1aaf6f0577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:21.984525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b085-5cbd-4b2e-97a3-6f1aaf6f0577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:21.985021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b085-5cbd-4b2e-97a3-6f1aaf6f0577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:21.985577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b085-5cbd-4b2e-97a3-6f1aaf6f0577 admin admin] PATH_INFO: `/v3/users/e74556db47b44db1bf4086f4711dc50c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:21.988048 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5b085-5cbd-4b2e-97a3-6f1aaf6f0577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e74556db47b44db1bf4086f4711dc50c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:21.996602 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5b085-5cbd-4b2e-97a3-6f1aaf6f0577 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:22.029117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:22.057293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c670feec-cc2c-4481-8788-9005013f3623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ef6a9c1c98146e181aefee41790b4ff', 'request_id': 'req-c670feec-cc2c-4481-8788-9005013f3623'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:22.077035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c670feec-cc2c-4481-8788-9005013f3623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ef6a9c1c98146e181aefee41790b4ff', 'request_id': 'req-c670feec-cc2c-4481-8788-9005013f3623'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:22.088002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c670feec-cc2c-4481-8788-9005013f3623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ef6a9c1c98146e181aefee41790b4ff', 'request_id': 'req-c670feec-cc2c-4481-8788-9005013f3623'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:22.096258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ecd5b085-5cbd-4b2e-97a3-6f1aaf6f0577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e74556db47b44db1bf4086f4711dc50c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:22.118542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c670feec-cc2c-4481-8788-9005013f3623 admin admin] The token cache is being invalidate because project 1ef6a9c1c98146e181aefee41790b4ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:22.119908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c670feec-cc2c-4481-8788-9005013f3623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:22.126437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ecd5b085-5cbd-4b2e-97a3-6f1aaf6f0577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e74556db47b44db1bf4086f4711dc50c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:22.127684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c670feec-cc2c-4481-8788-9005013f3623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:22.136616 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 929/1862] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:21 2022] DELETE /identity/v3/projects/1ef6a9c1c98146e181aefee41790b4ff => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:22.147458 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ecd5b085-5cbd-4b2e-97a3-6f1aaf6f0577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e74556db47b44db1bf4086f4711dc50c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:22.149968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ecd5b085-5cbd-4b2e-97a3-6f1aaf6f0577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e74556db47b44db1bf4086f4711dc50c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:22.158272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a5769-db90-448f-8ba4-2a04ccac0d58 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:22.158968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a5769-db90-448f-8ba4-2a04ccac0d58 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:22.160325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a5769-db90-448f-8ba4-2a04ccac0d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:22.196416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-997a5769-db90-448f-8ba4-2a04ccac0d58 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:22.199129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-997a5769-db90-448f-8ba4-2a04ccac0d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:22.245835 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ecd5b085-5cbd-4b2e-97a3-6f1aaf6f0577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e74556db47b44db1bf4086f4711dc50c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:22.278421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 934/1863] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:21 2022] DELETE /identity/v3/users/e74556db47b44db1bf4086f4711dc50c => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:22.365573 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 930/1864] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:22.379589 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c73c18-7b85-4ea3-8012-eb37e5ed16be None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:22.379906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c73c18-7b85-4ea3-8012-eb37e5ed16be None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:22.380238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c73c18-7b85-4ea3-8012-eb37e5ed16be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:22.458518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a0c73c18-7b85-4ea3-8012-eb37e5ed16be None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:22.460669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a0c73c18-7b85-4ea3-8012-eb37e5ed16be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:22.614434 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 935/1865] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:22.619115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a5769-db90-448f-8ba4-2a04ccac0d58 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:22.640445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a5769-db90-448f-8ba4-2a04ccac0d58 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:22.655422 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f649eb-cb98-4c0e-bddd-73a6a5afb2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:22.656145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f649eb-cb98-4c0e-bddd-73a6a5afb2cc admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:22.656425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f649eb-cb98-4c0e-bddd-73a6a5afb2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:22.656698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f649eb-cb98-4c0e-bddd-73a6a5afb2cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:22.657552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f649eb-cb98-4c0e-bddd-73a6a5afb2cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:04:22.657912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f649eb-cb98-4c0e-bddd-73a6a5afb2cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:22.660572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f649eb-cb98-4c0e-bddd-73a6a5afb2cc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:22.679161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 931/1866] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:04:22.686533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c73c18-7b85-4ea3-8012-eb37e5ed16be None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:22.703743 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c73c18-7b85-4ea3-8012-eb37e5ed16be None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:22.721220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62329984-4e87-4413-80e9-e49458262600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:22.721964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62329984-4e87-4413-80e9-e49458262600 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:22.723502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62329984-4e87-4413-80e9-e49458262600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:22.724021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62329984-4e87-4413-80e9-e49458262600 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:22.725130 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62329984-4e87-4413-80e9-e49458262600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:22.728423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62329984-4e87-4413-80e9-e49458262600 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:22.787166 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 936/1867] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:22 2022] POST /identity/v3/projects => generated 361 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:22.788442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f649eb-cb98-4c0e-bddd-73a6a5afb2cc admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:22.812546 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f649eb-cb98-4c0e-bddd-73a6a5afb2cc admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:22.848278 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:22.964786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a307f-f8cb-472f-a8a5-79f8ec5697d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:22.965601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a307f-f8cb-472f-a8a5-79f8ec5697d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:22.965989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a307f-f8cb-472f-a8a5-79f8ec5697d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:22.966343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a307f-f8cb-472f-a8a5-79f8ec5697d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:22.967167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a307f-f8cb-472f-a8a5-79f8ec5697d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:22.970298 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a307f-f8cb-472f-a8a5-79f8ec5697d4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:23.038406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 932/1868] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:22 2022] POST /identity/v3/users => generated 281 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:23.043412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62329984-4e87-4413-80e9-e49458262600 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:23.086186 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62329984-4e87-4413-80e9-e49458262600 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:23.099753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62641611-9a15-4cb1-9070-e6afde71ed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:23.100782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62641611-9a15-4cb1-9070-e6afde71ed51 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:23.101172 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62641611-9a15-4cb1-9070-e6afde71ed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:23.101675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62641611-9a15-4cb1-9070-e6afde71ed51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:23.102595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62641611-9a15-4cb1-9070-e6afde71ed51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:23.108271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62641611-9a15-4cb1-9070-e6afde71ed51 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:23.136856 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 937/1869] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:23.143950 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a307f-f8cb-472f-a8a5-79f8ec5697d4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:23.163247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62641611-9a15-4cb1-9070-e6afde71ed51 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:23.164440 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-62641611-9a15-4cb1-9070-e6afde71ed51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:23.166073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a307f-f8cb-472f-a8a5-79f8ec5697d4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:23.171715 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdbd6b-825f-437f-a243-25772a133abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:23.172596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdbd6b-825f-437f-a243-25772a133abd admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:23.172986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdbd6b-825f-437f-a243-25772a133abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:23.173425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdbd6b-825f-437f-a243-25772a133abd admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:23.174343 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afdbd6b-825f-437f-a243-25772a133abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:23.176710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afdbd6b-825f-437f-a243-25772a133abd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:23.189097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62641611-9a15-4cb1-9070-e6afde71ed51 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:23.198541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 933/1870] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:23.202679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdbd6b-825f-437f-a243-25772a133abd admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:23.215626 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684da63-c74b-47d4-9320-1e56067ed480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:23.216581 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684da63-c74b-47d4-9320-1e56067ed480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:23.216854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684da63-c74b-47d4-9320-1e56067ed480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:23.217140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684da63-c74b-47d4-9320-1e56067ed480 admin admin] PATH_INFO: `/v3/projects/bf36e39a21d6497ea08c85e08fc9ceab` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:23.218640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684da63-c74b-47d4-9320-1e56067ed480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf36e39a21d6497ea08c85e08fc9ceab)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:23.221091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684da63-c74b-47d4-9320-1e56067ed480 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:23.228422 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdbd6b-825f-437f-a243-25772a133abd admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:23.234070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8f567-8fa5-41b0-b0f1-33222009fb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:23.236214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8f567-8fa5-41b0-b0f1-33222009fb46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:23.236650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8f567-8fa5-41b0-b0f1-33222009fb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:23.237022 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8f567-8fa5-41b0-b0f1-33222009fb46 admin admin] PATH_INFO: `/v3/projects/86bf11ff2ed640ff8eb2bdbb2c31d4d7/users/f8167def85b74c7d8daa671c502eafdb/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:23.259756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8f567-8fa5-41b0-b0f1-33222009fb46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bf11ff2ed640ff8eb2bdbb2c31d4d7, user_id=f8167def85b74c7d8daa671c502eafdb, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:23.263076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8f567-8fa5-41b0-b0f1-33222009fb46 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:23.311166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 934/1871] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:23 2022] PUT /identity/v3/projects/86bf11ff2ed640ff8eb2bdbb2c31d4d7/users/f8167def85b74c7d8daa671c502eafdb/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:23.325080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c2c46-5f05-4e6c-b62c-5482ebd75361 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:23.325701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c2c46-5f05-4e6c-b62c-5482ebd75361 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:23.326320 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c2c46-5f05-4e6c-b62c-5482ebd75361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:23.359188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1684da63-c74b-47d4-9320-1e56067ed480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf36e39a21d6497ea08c85e08fc9ceab', 'request_id': 'req-1684da63-c74b-47d4-9320-1e56067ed480'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:23.372503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:23.381408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1684da63-c74b-47d4-9320-1e56067ed480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf36e39a21d6497ea08c85e08fc9ceab', 'request_id': 'req-1684da63-c74b-47d4-9320-1e56067ed480'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:23.396981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1684da63-c74b-47d4-9320-1e56067ed480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf36e39a21d6497ea08c85e08fc9ceab', 'request_id': 'req-1684da63-c74b-47d4-9320-1e56067ed480'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:23.399055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-833c2c46-5f05-4e6c-b62c-5482ebd75361 None None] MFA Rules not processed for user `f8167def85b74c7d8daa671c502eafdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:23.400645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-833c2c46-5f05-4e6c-b62c-5482ebd75361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:23.434164 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1684da63-c74b-47d4-9320-1e56067ed480 admin admin] The token cache is being invalidate because project bf36e39a21d6497ea08c85e08fc9ceab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:23.434456 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1684da63-c74b-47d4-9320-1e56067ed480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:23.437291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1684da63-c74b-47d4-9320-1e56067ed480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:23.439195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 938/1872] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:23 2022] DELETE /identity/v3/projects/bf36e39a21d6497ea08c85e08fc9ceab => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:23.446588 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9aa16-cfc9-4488-b984-8dc12b5f5fb6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:23.460217 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9aa16-cfc9-4488-b984-8dc12b5f5fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:23.460792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9aa16-cfc9-4488-b984-8dc12b5f5fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:23.506968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-56c9aa16-cfc9-4488-b984-8dc12b5f5fb6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:23.509394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-56c9aa16-cfc9-4488-b984-8dc12b5f5fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:23.633533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 935/1873] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:23.668514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 939/1874] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:23.677325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58258c73-5316-4713-bc79-462f676130c3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:23.677797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58258c73-5316-4713-bc79-462f676130c3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:23.678297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58258c73-5316-4713-bc79-462f676130c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:23.717869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-58258c73-5316-4713-bc79-462f676130c3 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:23.719245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-58258c73-5316-4713-bc79-462f676130c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:23.725925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9aa16-cfc9-4488-b984-8dc12b5f5fb6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:23.727885 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-56c9aa16-cfc9-4488-b984-8dc12b5f5fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:23.758394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 936/1875] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:23.760514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58258c73-5316-4713-bc79-462f676130c3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:23.783787 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58258c73-5316-4713-bc79-462f676130c3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:23.794379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276dd7e-5998-49e0-9109-e1712214738c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:23.798066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276dd7e-5998-49e0-9109-e1712214738c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:23.798066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276dd7e-5998-49e0-9109-e1712214738c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:23.798066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276dd7e-5998-49e0-9109-e1712214738c admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:23.798066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276dd7e-5998-49e0-9109-e1712214738c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:04:23.798066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276dd7e-5998-49e0-9109-e1712214738c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:23.799541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276dd7e-5998-49e0-9109-e1712214738c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:23.824444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 937/1876] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:04:23.825520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276dd7e-5998-49e0-9109-e1712214738c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:23.840876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276dd7e-5998-49e0-9109-e1712214738c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:23.849112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3709198-2a9c-4a29-91d0-9c645cf45bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:23.849873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3709198-2a9c-4a29-91d0-9c645cf45bed admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:23.850163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3709198-2a9c-4a29-91d0-9c645cf45bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:23.850445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3709198-2a9c-4a29-91d0-9c645cf45bed admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:23.851190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3709198-2a9c-4a29-91d0-9c645cf45bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:23.856598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3709198-2a9c-4a29-91d0-9c645cf45bed admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:23.875497 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9aa16-cfc9-4488-b984-8dc12b5f5fb6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:23.890739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05527ebb-cd1d-413b-b483-54eb85efcacd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:23.891524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05527ebb-cd1d-413b-b483-54eb85efcacd service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:23.891827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05527ebb-cd1d-413b-b483-54eb85efcacd service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:23.892110 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05527ebb-cd1d-413b-b483-54eb85efcacd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:23.903211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05527ebb-cd1d-413b-b483-54eb85efcacd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:23.905078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05527ebb-cd1d-413b-b483-54eb85efcacd service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:23.929174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 938/1877] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:23 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:23.937719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3709198-2a9c-4a29-91d0-9c645cf45bed admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:23.944195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 940/1878] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:23.967484 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05527ebb-cd1d-413b-b483-54eb85efcacd service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:23.969200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-05527ebb-cd1d-413b-b483-54eb85efcacd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:24.003549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05527ebb-cd1d-413b-b483-54eb85efcacd service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:24.021108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:24.143112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3709198-2a9c-4a29-91d0-9c645cf45bed admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:24.149167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa7d4f-22e4-4ad4-8546-e102229442a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:24.150084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa7d4f-22e4-4ad4-8546-e102229442a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:24.150376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa7d4f-22e4-4ad4-8546-e102229442a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:24.150661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa7d4f-22e4-4ad4-8546-e102229442a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:24.151408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa7d4f-22e4-4ad4-8546-e102229442a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:24.153586 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa7d4f-22e4-4ad4-8546-e102229442a6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:24.158204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636607c-54c8-4f79-b911-6e804bf7aa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:24.159484 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636607c-54c8-4f79-b911-6e804bf7aa26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:24.159815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636607c-54c8-4f79-b911-6e804bf7aa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:24.160099 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636607c-54c8-4f79-b911-6e804bf7aa26 admin admin] PATH_INFO: `/v3/users/f8167def85b74c7d8daa671c502eafdb` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:24.164361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636607c-54c8-4f79-b911-6e804bf7aa26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8167def85b74c7d8daa671c502eafdb)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:24.173774 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636607c-54c8-4f79-b911-6e804bf7aa26 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:24.182866 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 939/1879] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:23 2022] POST /identity/v3/users => generated 288 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:24.188486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa7d4f-22e4-4ad4-8546-e102229442a6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:24.221940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa7d4f-22e4-4ad4-8546-e102229442a6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:24.232629 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e54dfb-46b6-41a9-a473-3f6b12ad2681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:24.233966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e54dfb-46b6-41a9-a473-3f6b12ad2681 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:24.234511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e54dfb-46b6-41a9-a473-3f6b12ad2681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:24.235344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e54dfb-46b6-41a9-a473-3f6b12ad2681 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:24.237054 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e54dfb-46b6-41a9-a473-3f6b12ad2681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:24.241675 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e54dfb-46b6-41a9-a473-3f6b12ad2681 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:24.243669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:24.284540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 940/1880] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:24.285576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e54dfb-46b6-41a9-a473-3f6b12ad2681 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:24.312334 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a636607c-54c8-4f79-b911-6e804bf7aa26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8167def85b74c7d8daa671c502eafdb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:24.338425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a636607c-54c8-4f79-b911-6e804bf7aa26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8167def85b74c7d8daa671c502eafdb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:24.359403 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a636607c-54c8-4f79-b911-6e804bf7aa26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8167def85b74c7d8daa671c502eafdb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:24.361687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a636607c-54c8-4f79-b911-6e804bf7aa26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8167def85b74c7d8daa671c502eafdb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:24.382841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e54dfb-46b6-41a9-a473-3f6b12ad2681 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:24.385078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a636607c-54c8-4f79-b911-6e804bf7aa26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8167def85b74c7d8daa671c502eafdb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:24.390173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb83d1f-a1cf-423f-a323-ef3a6653efe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:24.391506 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb83d1f-a1cf-423f-a323-ef3a6653efe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:24.392187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb83d1f-a1cf-423f-a323-ef3a6653efe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:24.392702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb83d1f-a1cf-423f-a323-ef3a6653efe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:24.393687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 941/1881] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:23 2022] DELETE /identity/v3/users/f8167def85b74c7d8daa671c502eafdb => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:24.398437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c277d8-5271-41e0-a8c0-a11e381de26b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:24.398570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb83d1f-a1cf-423f-a323-ef3a6653efe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:24.398980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c277d8-5271-41e0-a8c0-a11e381de26b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:24.399361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c277d8-5271-41e0-a8c0-a11e381de26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:24.402054 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb83d1f-a1cf-423f-a323-ef3a6653efe6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:24.428665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b8c277d8-5271-41e0-a8c0-a11e381de26b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:24.430652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 941/1882] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:24.430948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b8c277d8-5271-41e0-a8c0-a11e381de26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:24.432018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb83d1f-a1cf-423f-a323-ef3a6653efe6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:24.434105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9bb83d1f-a1cf-423f-a323-ef3a6653efe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:24.471216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb83d1f-a1cf-423f-a323-ef3a6653efe6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:24.517456 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 942/1883] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:24.529462 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c277d8-5271-41e0-a8c0-a11e381de26b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:24.530952 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b8c277d8-5271-41e0-a8c0-a11e381de26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:24.587363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf38ea1-4e7c-43d2-b1df-dfc135159a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:24.588355 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf38ea1-4e7c-43d2-b1df-dfc135159a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:24.588643 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf38ea1-4e7c-43d2-b1df-dfc135159a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:24.588948 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf38ea1-4e7c-43d2-b1df-dfc135159a11 admin admin] PATH_INFO: `/v3/projects/bda6d17f355e47c9a675ce3bf595bf15/users/2b48cf33019b4e98a7ac34b66e7e5652/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:24.614230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf38ea1-4e7c-43d2-b1df-dfc135159a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda6d17f355e47c9a675ce3bf595bf15, user_id=2b48cf33019b4e98a7ac34b66e7e5652, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:24.617514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf38ea1-4e7c-43d2-b1df-dfc135159a11 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:24.628628 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c277d8-5271-41e0-a8c0-a11e381de26b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:24.634611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 942/1884] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:24 2022] PUT /identity/v3/projects/bda6d17f355e47c9a675ce3bf595bf15/users/2b48cf33019b4e98a7ac34b66e7e5652/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:24.646414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6c1e6-0703-480c-bc94-34c57b755c56 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:24.646682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6c1e6-0703-480c-bc94-34c57b755c56 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:24.646961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6c1e6-0703-480c-bc94-34c57b755c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:24.684784 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cac6c1e6-0703-480c-bc94-34c57b755c56 None None] MFA Rules not processed for user `2b48cf33019b4e98a7ac34b66e7e5652`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:24.686115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cac6c1e6-0703-480c-bc94-34c57b755c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:24.702861 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81e5c8-1401-4947-8307-92d1f99fce3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:24.704039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e5c8-1401-4947-8307-92d1f99fce3d service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:24.704640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e5c8-1401-4947-8307-92d1f99fce3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:24.704993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e5c8-1401-4947-8307-92d1f99fce3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:24.747462 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:24.819123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:24.833813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 943/1885] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:24.847512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6c1e6-0703-480c-bc94-34c57b755c56 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:24.849820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cac6c1e6-0703-480c-bc94-34c57b755c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:24.859988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81e5c8-1401-4947-8307-92d1f99fce3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:24.864464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81e5c8-1401-4947-8307-92d1f99fce3d service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:24.883609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 943/1886] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:04:24.953280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6c1e6-0703-480c-bc94-34c57b755c56 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:24.967003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a466b-0a98-4baa-8638-67d834763830 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a122422a45446c9a63d67b4f931fa47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:24.968090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a466b-0a98-4baa-8638-67d834763830 service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:24.968489 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a466b-0a98-4baa-8638-67d834763830 service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:24.971183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a466b-0a98-4baa-8638-67d834763830 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:24.986260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a466b-0a98-4baa-8638-67d834763830 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:24.989932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a466b-0a98-4baa-8638-67d834763830 service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:25.012068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 944/1887] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:25.738341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81e5c8-1401-4947-8307-92d1f99fce3d service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:25.740700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ed81e5c8-1401-4947-8307-92d1f99fce3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:25.794721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81e5c8-1401-4947-8307-92d1f99fce3d service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:25.812728 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:25.818682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:25.819124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:25.819453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3 admin admin] PATH_INFO: `/v3/projects/86bf11ff2ed640ff8eb2bdbb2c31d4d7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:25.821721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86bf11ff2ed640ff8eb2bdbb2c31d4d7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:25.827818 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:25.889234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a466b-0a98-4baa-8638-67d834763830 service glance] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:25.894059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-587a466b-0a98-4baa-8638-67d834763830 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:25.935416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a466b-0a98-4baa-8638-67d834763830 service glance] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:25.953898 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8abc9-f0ca-4812-bf00-83a9ba6fa2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:25.955601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8abc9-f0ca-4812-bf00-83a9ba6fa2ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:25.956096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8abc9-f0ca-4812-bf00-83a9ba6fa2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:25.956589 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8abc9-f0ca-4812-bf00-83a9ba6fa2ae admin admin] PATH_INFO: `/v3/users/817b662624924698ab7b18d7d264faa6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:25.959975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8abc9-f0ca-4812-bf00-83a9ba6fa2ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=817b662624924698ab7b18d7d264faa6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:25.963154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8abc9-f0ca-4812-bf00-83a9ba6fa2ae admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:25.990488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86bf11ff2ed640ff8eb2bdbb2c31d4d7', 'request_id': 'req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:26.004247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86bf11ff2ed640ff8eb2bdbb2c31d4d7', 'request_id': 'req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:26.024046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86bf11ff2ed640ff8eb2bdbb2c31d4d7', 'request_id': 'req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:26.041944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3 admin admin] The token cache is being invalidate because project 86bf11ff2ed640ff8eb2bdbb2c31d4d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:26.042569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:26.043399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-da55e16a-abbd-4afb-be8d-3cd87ea62bd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:26.047305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 944/1888] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:25 2022] DELETE /identity/v3/projects/86bf11ff2ed640ff8eb2bdbb2c31d4d7 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:26.052263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b436b-7f4d-4d25-9baa-944e5c5b6995 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:26.052549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b436b-7f4d-4d25-9baa-944e5c5b6995 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:26.052826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b436b-7f4d-4d25-9baa-944e5c5b6995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:26.056569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d1f8abc9-f0ca-4812-bf00-83a9ba6fa2ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '817b662624924698ab7b18d7d264faa6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:26.057433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d1f8abc9-f0ca-4812-bf00-83a9ba6fa2ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '817b662624924698ab7b18d7d264faa6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:26.059432 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d1f8abc9-f0ca-4812-bf00-83a9ba6fa2ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '817b662624924698ab7b18d7d264faa6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:26.088430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d1f8abc9-f0ca-4812-bf00-83a9ba6fa2ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '817b662624924698ab7b18d7d264faa6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:26.109848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9d5b436b-7f4d-4d25-9baa-944e5c5b6995 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:26.135177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9d5b436b-7f4d-4d25-9baa-944e5c5b6995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:26.158103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d1f8abc9-f0ca-4812-bf00-83a9ba6fa2ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '817b662624924698ab7b18d7d264faa6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:26.192311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 945/1889] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:25 2022] DELETE /identity/v3/users/817b662624924698ab7b18d7d264faa6 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:26.197204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dbb4f-8f90-4c25-9e38-7efe91feec5a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:26.197672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dbb4f-8f90-4c25-9e38-7efe91feec5a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:26.198097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dbb4f-8f90-4c25-9e38-7efe91feec5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:26.225008 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-730dbb4f-8f90-4c25-9e38-7efe91feec5a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:26.237001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-730dbb4f-8f90-4c25-9e38-7efe91feec5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:26.350654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 946/1890] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:26.351118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 945/1891] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:26.354528 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8a7cb-8d2d-441d-92fc-ed541145c8e9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:26.354814 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8a7cb-8d2d-441d-92fc-ed541145c8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:26.355091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8a7cb-8d2d-441d-92fc-ed541145c8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:26.358296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b436b-7f4d-4d25-9baa-944e5c5b6995 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:26.360426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9d5b436b-7f4d-4d25-9baa-944e5c5b6995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:26.397532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9bf8a7cb-8d2d-441d-92fc-ed541145c8e9 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:26.398917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9bf8a7cb-8d2d-441d-92fc-ed541145c8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:26.413684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 947/1892] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:26.415849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8a7cb-8d2d-441d-92fc-ed541145c8e9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:26.426396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8a7cb-8d2d-441d-92fc-ed541145c8e9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:26.433741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde244c-82e3-42c6-beda-eed62d0aa8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:26.434641 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde244c-82e3-42c6-beda-eed62d0aa8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:26.435230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde244c-82e3-42c6-beda-eed62d0aa8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:26.435230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde244c-82e3-42c6-beda-eed62d0aa8d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:26.436105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde244c-82e3-42c6-beda-eed62d0aa8d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:04:26.436502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde244c-82e3-42c6-beda-eed62d0aa8d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:26.439040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde244c-82e3-42c6-beda-eed62d0aa8d6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:26.439225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:26.441108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:26.460392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 948/1893] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:04:26.464159 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde244c-82e3-42c6-beda-eed62d0aa8d6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:26.477728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde244c-82e3-42c6-beda-eed62d0aa8d6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:26.478460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b436b-7f4d-4d25-9baa-944e5c5b6995 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:26.483330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881baa3-f1e3-46b6-b8c6-7e1eed60da46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:26.484194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881baa3-f1e3-46b6-b8c6-7e1eed60da46 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:26.484644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881baa3-f1e3-46b6-b8c6-7e1eed60da46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:26.484931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881baa3-f1e3-46b6-b8c6-7e1eed60da46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:26.485682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881baa3-f1e3-46b6-b8c6-7e1eed60da46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:26.487902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881baa3-f1e3-46b6-b8c6-7e1eed60da46 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:26.523936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 949/1894] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:26 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:04:26.530748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881baa3-f1e3-46b6-b8c6-7e1eed60da46 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:26.552792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0d5cc-55e8-4d9b-b3f0-e4538363520b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:26.554925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0d5cc-55e8-4d9b-b3f0-e4538363520b service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:26.554925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0d5cc-55e8-4d9b-b3f0-e4538363520b service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:26.554925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0d5cc-55e8-4d9b-b3f0-e4538363520b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:26.567790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0d5cc-55e8-4d9b-b3f0-e4538363520b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:26.570127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0d5cc-55e8-4d9b-b3f0-e4538363520b service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:26.584441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 946/1895] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:26.641650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881baa3-f1e3-46b6-b8c6-7e1eed60da46 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:26.650502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551726e-8c6a-4731-a6f1-22c62b5f5c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:26.651274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551726e-8c6a-4731-a6f1-22c62b5f5c66 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:26.651619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551726e-8c6a-4731-a6f1-22c62b5f5c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:26.651962 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551726e-8c6a-4731-a6f1-22c62b5f5c66 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:26.652781 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551726e-8c6a-4731-a6f1-22c62b5f5c66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:26.656299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551726e-8c6a-4731-a6f1-22c62b5f5c66 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:26.706235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 950/1896] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:26 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:26.709579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0d5cc-55e8-4d9b-b3f0-e4538363520b service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:26.749008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0d5cc-55e8-4d9b-b3f0-e4538363520b service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:26.757299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74a199-5beb-4b22-a252-50bad51f44d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:26.758302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74a199-5beb-4b22-a252-50bad51f44d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:26.758764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74a199-5beb-4b22-a252-50bad51f44d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:26.759147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74a199-5beb-4b22-a252-50bad51f44d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:26.760226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74a199-5beb-4b22-a252-50bad51f44d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:26.762560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74a199-5beb-4b22-a252-50bad51f44d2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:26.776265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 947/1897] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:26.780832 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551726e-8c6a-4731-a6f1-22c62b5f5c66 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:26.792670 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551726e-8c6a-4731-a6f1-22c62b5f5c66 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:26.798025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd9e3f-b080-4cdf-b300-cfaa93750b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:26.798869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd9e3f-b080-4cdf-b300-cfaa93750b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:26.799025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd9e3f-b080-4cdf-b300-cfaa93750b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:26.799305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd9e3f-b080-4cdf-b300-cfaa93750b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:26.800157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd9e3f-b080-4cdf-b300-cfaa93750b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:26.802905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd9e3f-b080-4cdf-b300-cfaa93750b79 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:26.822381 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 951/1898] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:26.828664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74a199-5beb-4b22-a252-50bad51f44d2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:26.859057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74a199-5beb-4b22-a252-50bad51f44d2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:26.869775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12236229-0aad-4222-a631-cbede0a58f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:26.871125 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12236229-0aad-4222-a631-cbede0a58f6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:26.871651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12236229-0aad-4222-a631-cbede0a58f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:26.872173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12236229-0aad-4222-a631-cbede0a58f6a admin admin] PATH_INFO: `/v3/projects/f35224175d174ef7996a7ad382015018/users/1fff94ed9c6946b9a2f6c138c8b6dee4/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:26.894860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12236229-0aad-4222-a631-cbede0a58f6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f35224175d174ef7996a7ad382015018, user_id=1fff94ed9c6946b9a2f6c138c8b6dee4, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:26.898729 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12236229-0aad-4222-a631-cbede0a58f6a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:26.947158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 948/1899] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:26 2022] PUT /identity/v3/projects/f35224175d174ef7996a7ad382015018/users/1fff94ed9c6946b9a2f6c138c8b6dee4/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:26.969947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbaac5-9964-423b-ae6c-bcfa8da37db1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:26.970697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbaac5-9964-423b-ae6c-bcfa8da37db1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:26.970844 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbaac5-9964-423b-ae6c-bcfa8da37db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:27.036384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2efbaac5-9964-423b-ae6c-bcfa8da37db1 None None] MFA Rules not processed for user `1fff94ed9c6946b9a2f6c138c8b6dee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:27.038591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2efbaac5-9964-423b-ae6c-bcfa8da37db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:27.245733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 952/1900] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:27.261603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12236229-0aad-4222-a631-cbede0a58f6a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:27.265548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-12236229-0aad-4222-a631-cbede0a58f6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:27.290166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbaac5-9964-423b-ae6c-bcfa8da37db1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:27.292246 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2efbaac5-9964-423b-ae6c-bcfa8da37db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:27.415457 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12236229-0aad-4222-a631-cbede0a58f6a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:27.429596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7cccd-3b4b-4160-93fe-f211e62744f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:27.430629 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7cccd-3b4b-4160-93fe-f211e62744f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:27.431071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7cccd-3b4b-4160-93fe-f211e62744f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:27.431681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7cccd-3b4b-4160-93fe-f211e62744f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:27.442342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:27.445086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7cccd-3b4b-4160-93fe-f211e62744f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:27.448252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7cccd-3b4b-4160-93fe-f211e62744f6 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:27.481133 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 949/1901] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:27.579157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbaac5-9964-423b-ae6c-bcfa8da37db1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:27.594897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27dfda-181d-45be-b9ae-88d72f187f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:27.596006 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27dfda-181d-45be-b9ae-88d72f187f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:27.596433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27dfda-181d-45be-b9ae-88d72f187f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:27.596781 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27dfda-181d-45be-b9ae-88d72f187f70 admin admin] PATH_INFO: `/v3/projects/4f565185e72d4317b9bdd7d2d4c7778d` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:27.607677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27dfda-181d-45be-b9ae-88d72f187f70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f565185e72d4317b9bdd7d2d4c7778d)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:27.609900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27dfda-181d-45be-b9ae-88d72f187f70 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:27.729896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7b27dfda-181d-45be-b9ae-88d72f187f70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f565185e72d4317b9bdd7d2d4c7778d', 'request_id': 'req-7b27dfda-181d-45be-b9ae-88d72f187f70'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:27.742614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7b27dfda-181d-45be-b9ae-88d72f187f70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f565185e72d4317b9bdd7d2d4c7778d', 'request_id': 'req-7b27dfda-181d-45be-b9ae-88d72f187f70'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:27.761669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7b27dfda-181d-45be-b9ae-88d72f187f70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f565185e72d4317b9bdd7d2d4c7778d', 'request_id': 'req-7b27dfda-181d-45be-b9ae-88d72f187f70'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:27.823224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7b27dfda-181d-45be-b9ae-88d72f187f70 admin admin] The token cache is being invalidate because project 4f565185e72d4317b9bdd7d2d4c7778d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:27.823224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7b27dfda-181d-45be-b9ae-88d72f187f70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:27.824360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7b27dfda-181d-45be-b9ae-88d72f187f70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:27.830734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 953/1902] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:27 2022] DELETE /identity/v3/projects/4f565185e72d4317b9bdd7d2d4c7778d => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:27.834454 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7cccd-3b4b-4160-93fe-f211e62744f6 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:27.876431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7cccd-3b4b-4160-93fe-f211e62744f6 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:28.090077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:28.091144 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:28.091535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:28.091927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] PATH_INFO: `/v3/users/a0e254343d2b4ea1a05b2df64c3340ad` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:28.094983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e254343d2b4ea1a05b2df64c3340ad)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:28.102008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:28.111602 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:28.170886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35d932-3d84-42d6-9244-539fe84487fb None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:28.171771 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35d932-3d84-42d6-9244-539fe84487fb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:28.172355 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35d932-3d84-42d6-9244-539fe84487fb None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:28.176856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 954/1903] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 08:04:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:04:28.199035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a40ab-a714-4a69-bfca-2dd9ddc62090 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:28.199512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a40ab-a714-4a69-bfca-2dd9ddc62090 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:28.199973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a40ab-a714-4a69-bfca-2dd9ddc62090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:28.260170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0c2a40ab-a714-4a69-bfca-2dd9ddc62090 None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:28.261885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0c2a40ab-a714-4a69-bfca-2dd9ddc62090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:28.281200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e254343d2b4ea1a05b2df64c3340ad'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:28.323130 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e254343d2b4ea1a05b2df64c3340ad'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:28.359001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e254343d2b4ea1a05b2df64c3340ad'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:28.360850 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e254343d2b4ea1a05b2df64c3340ad'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:28.412063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e254343d2b4ea1a05b2df64c3340ad'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:28.430761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 955/1904] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 08:04:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:28.435319 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 950/1905] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:27 2022] DELETE /identity/v3/users/a0e254343d2b4ea1a05b2df64c3340ad => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:28.463060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a40ab-a714-4a69-bfca-2dd9ddc62090 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:28.466863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0c2a40ab-a714-4a69-bfca-2dd9ddc62090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:28.501053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a40ab-a714-4a69-bfca-2dd9ddc62090 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:28.623398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:28.635822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71168396-b5ba-40c3-a13e-57e942b3b9b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:28.636673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71168396-b5ba-40c3-a13e-57e942b3b9b6 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:28.637545 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71168396-b5ba-40c3-a13e-57e942b3b9b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:28.637956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71168396-b5ba-40c3-a13e-57e942b3b9b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:28.650205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71168396-b5ba-40c3-a13e-57e942b3b9b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:28.651776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71168396-b5ba-40c3-a13e-57e942b3b9b6 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:28.664899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 956/1906] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:29.121986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:29.125158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:29.253507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a8705-d620-4f02-81f9-01fbeac5736e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:29.258906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49736b-13bf-4d4a-9071-065c3ced94e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:29.259883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49736b-13bf-4d4a-9071-065c3ced94e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:29.260212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49736b-13bf-4d4a-9071-065c3ced94e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:29.260504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49736b-13bf-4d4a-9071-065c3ced94e0 admin admin] PATH_INFO: `/v3/projects/c9a5c75a0f8e449fa90faf132320ac5a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:29.262211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49736b-13bf-4d4a-9071-065c3ced94e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9a5c75a0f8e449fa90faf132320ac5a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:29.264217 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49736b-13bf-4d4a-9071-065c3ced94e0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:29.347494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df49736b-13bf-4d4a-9071-065c3ced94e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9a5c75a0f8e449fa90faf132320ac5a', 'request_id': 'req-df49736b-13bf-4d4a-9071-065c3ced94e0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:29.356961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df49736b-13bf-4d4a-9071-065c3ced94e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9a5c75a0f8e449fa90faf132320ac5a', 'request_id': 'req-df49736b-13bf-4d4a-9071-065c3ced94e0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:29.368094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df49736b-13bf-4d4a-9071-065c3ced94e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9a5c75a0f8e449fa90faf132320ac5a', 'request_id': 'req-df49736b-13bf-4d4a-9071-065c3ced94e0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:29.392229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df49736b-13bf-4d4a-9071-065c3ced94e0 admin admin] The token cache is being invalidate because project c9a5c75a0f8e449fa90faf132320ac5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:29.392831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df49736b-13bf-4d4a-9071-065c3ced94e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:29.393961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df49736b-13bf-4d4a-9071-065c3ced94e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:29.399588 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 951/1907] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:29 2022] DELETE /identity/v3/projects/c9a5c75a0f8e449fa90faf132320ac5a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:29.402298 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78b891-5ced-4980-863a-a9bce7dcc66c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:29.403020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78b891-5ced-4980-863a-a9bce7dcc66c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:29.403207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78b891-5ced-4980-863a-a9bce7dcc66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:29.440560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5d78b891-5ced-4980-863a-a9bce7dcc66c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:29.443292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5d78b891-5ced-4980-863a-a9bce7dcc66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:29.562314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 957/1908] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:29.578924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b97b96-cd3e-48d7-ad4f-cecbfe303d99 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:29.579539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b97b96-cd3e-48d7-ad4f-cecbfe303d99 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:29.580547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b97b96-cd3e-48d7-ad4f-cecbfe303d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:29.625536 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f8b97b96-cd3e-48d7-ad4f-cecbfe303d99 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:29.634747 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f8b97b96-cd3e-48d7-ad4f-cecbfe303d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:29.652988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 952/1909] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:29.653488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78b891-5ced-4980-863a-a9bce7dcc66c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:29.668067 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78b891-5ced-4980-863a-a9bce7dcc66c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:29.684478 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cbe71-056e-4545-b027-fe3ab76fae01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:29.687709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cbe71-056e-4545-b027-fe3ab76fae01 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:29.688606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cbe71-056e-4545-b027-fe3ab76fae01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:29.695987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cbe71-056e-4545-b027-fe3ab76fae01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:29.705167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cbe71-056e-4545-b027-fe3ab76fae01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:04:29.706237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cbe71-056e-4545-b027-fe3ab76fae01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:29.716937 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cbe71-056e-4545-b027-fe3ab76fae01 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:29.747598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 958/1910] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:04:29.756160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b97b96-cd3e-48d7-ad4f-cecbfe303d99 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:29.776001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:29.785289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b97b96-cd3e-48d7-ad4f-cecbfe303d99 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:29.801299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d9a40-0b6e-41f4-b4a0-c3cdd28a4be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:29.802349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d9a40-0b6e-41f4-b4a0-c3cdd28a4be8 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:29.802785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d9a40-0b6e-41f4-b4a0-c3cdd28a4be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:29.803225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d9a40-0b6e-41f4-b4a0-c3cdd28a4be8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:29.804636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d9a40-0b6e-41f4-b4a0-c3cdd28a4be8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:29.808140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d9a40-0b6e-41f4-b4a0-c3cdd28a4be8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:29.872307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cbe71-056e-4545-b027-fe3ab76fae01 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:29.879338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 953/1911] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:29 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:29.898053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cbe71-056e-4545-b027-fe3ab76fae01 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:30.029938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56655db-4172-42e0-b514-cdfc11586ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:30.033146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56655db-4172-42e0-b514-cdfc11586ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:30.033146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56655db-4172-42e0-b514-cdfc11586ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:30.034266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56655db-4172-42e0-b514-cdfc11586ce3 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:30.036274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56655db-4172-42e0-b514-cdfc11586ce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:30.042682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56655db-4172-42e0-b514-cdfc11586ce3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:30.158302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 959/1912] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:29 2022] POST /identity/v3/users => generated 284 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:30.162129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d9a40-0b6e-41f4-b4a0-c3cdd28a4be8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:30.212868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d9a40-0b6e-41f4-b4a0-c3cdd28a4be8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:30.232709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a720a-c186-49ef-a37a-da6b03d74c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:30.236696 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a720a-c186-49ef-a37a-da6b03d74c5d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:30.237330 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a720a-c186-49ef-a37a-da6b03d74c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:30.238043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a720a-c186-49ef-a37a-da6b03d74c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:30.240706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a720a-c186-49ef-a37a-da6b03d74c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:30.247326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a720a-c186-49ef-a37a-da6b03d74c5d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:30.293390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 954/1913] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:04:30.301479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56655db-4172-42e0-b514-cdfc11586ce3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:30.325007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56655db-4172-42e0-b514-cdfc11586ce3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:30.333945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec5b31-8358-4dd6-b885-79bcc2d67305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:30.335506 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec5b31-8358-4dd6-b885-79bcc2d67305 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:30.336209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec5b31-8358-4dd6-b885-79bcc2d67305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:30.337821 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec5b31-8358-4dd6-b885-79bcc2d67305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:30.339539 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec5b31-8358-4dd6-b885-79bcc2d67305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:30.343765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec5b31-8358-4dd6-b885-79bcc2d67305 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:30.370137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 960/1914] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:30.378908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a720a-c186-49ef-a37a-da6b03d74c5d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:30.428513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a720a-c186-49ef-a37a-da6b03d74c5d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:30.436084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a6ae0-7419-4ecc-9ba5-4b949f5f37a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:30.437481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a6ae0-7419-4ecc-9ba5-4b949f5f37a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:30.437995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a6ae0-7419-4ecc-9ba5-4b949f5f37a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:30.438367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a6ae0-7419-4ecc-9ba5-4b949f5f37a2 admin admin] PATH_INFO: `/v3/projects/4a95d33c4ef74901bf320207aaf64fb0/users/5e6a87bdb4e949288963524b663edd9c/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:30.464010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a6ae0-7419-4ecc-9ba5-4b949f5f37a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a95d33c4ef74901bf320207aaf64fb0, user_id=5e6a87bdb4e949288963524b663edd9c, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:30.467317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a6ae0-7419-4ecc-9ba5-4b949f5f37a2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:30.499562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 955/1915] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:30 2022] PUT /identity/v3/projects/4a95d33c4ef74901bf320207aaf64fb0/users/5e6a87bdb4e949288963524b663edd9c/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:30.520910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825ff2f-493d-4cd6-a6dd-0add5e31d6b9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:30.521548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825ff2f-493d-4cd6-a6dd-0add5e31d6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:30.526932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825ff2f-493d-4cd6-a6dd-0add5e31d6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:30.625179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9825ff2f-493d-4cd6-a6dd-0add5e31d6b9 None None] MFA Rules not processed for user `5e6a87bdb4e949288963524b663edd9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:30.626530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9825ff2f-493d-4cd6-a6dd-0add5e31d6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:30.713376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:30.756163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 961/1916] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:30.758458 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a6ae0-7419-4ecc-9ba5-4b949f5f37a2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:30.900086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a6ae0-7419-4ecc-9ba5-4b949f5f37a2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:30.909168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba8e97-4e52-4871-8b06-c2c5561f3cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:30.910385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba8e97-4e52-4871-8b06-c2c5561f3cae admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:30.910914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba8e97-4e52-4871-8b06-c2c5561f3cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:30.911364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba8e97-4e52-4871-8b06-c2c5561f3cae admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:30.912623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba8e97-4e52-4871-8b06-c2c5561f3cae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:30.916143 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba8e97-4e52-4871-8b06-c2c5561f3cae admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:30.965050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 956/1917] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:30 2022] POST /identity/v3/projects => generated 367 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:30.969074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825ff2f-493d-4cd6-a6dd-0add5e31d6b9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:30.994492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825ff2f-493d-4cd6-a6dd-0add5e31d6b9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:31.116773 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5a41d-8f55-4b0c-b4e0-f17872bbb220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:31.118637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5a41d-8f55-4b0c-b4e0-f17872bbb220 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:31.119161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5a41d-8f55-4b0c-b4e0-f17872bbb220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:31.119573 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5a41d-8f55-4b0c-b4e0-f17872bbb220 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:31.121881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5a41d-8f55-4b0c-b4e0-f17872bbb220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:31.125121 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5a41d-8f55-4b0c-b4e0-f17872bbb220 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:31.161703 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 962/1918] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:30 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:31.166146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba8e97-4e52-4871-8b06-c2c5561f3cae admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:31.184832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba8e97-4e52-4871-8b06-c2c5561f3cae admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:31.192536 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c4edf-9511-4dd5-ab38-16c0530eb5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:31.193748 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c4edf-9511-4dd5-ab38-16c0530eb5de admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:31.194282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c4edf-9511-4dd5-ab38-16c0530eb5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:31.194808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c4edf-9511-4dd5-ab38-16c0530eb5de admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:31.196171 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c4edf-9511-4dd5-ab38-16c0530eb5de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:31.199732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c4edf-9511-4dd5-ab38-16c0530eb5de admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:31.221797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 957/1919] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:31.227877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5a41d-8f55-4b0c-b4e0-f17872bbb220 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:31.248153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5a41d-8f55-4b0c-b4e0-f17872bbb220 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:31.257181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effce6b-d455-44de-bd25-7369711f9eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:31.258684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effce6b-d455-44de-bd25-7369711f9eda admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:31.259225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effce6b-d455-44de-bd25-7369711f9eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:31.259772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effce6b-d455-44de-bd25-7369711f9eda admin admin] PATH_INFO: `/v3/projects/e88fd9ccc4e8464fa4120c1a6e15ebee/users/733afed26f8c448e82cc7c697789bc25/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:31.286316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effce6b-d455-44de-bd25-7369711f9eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=e88fd9ccc4e8464fa4120c1a6e15ebee, user_id=733afed26f8c448e82cc7c697789bc25, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:31.289115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effce6b-d455-44de-bd25-7369711f9eda admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:31.306282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 963/1920] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:31 2022] PUT /identity/v3/projects/e88fd9ccc4e8464fa4120c1a6e15ebee/users/733afed26f8c448e82cc7c697789bc25/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:31.310626 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c4edf-9511-4dd5-ab38-16c0530eb5de admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:31.321448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c4edf-9511-4dd5-ab38-16c0530eb5de admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:31.392801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e5299-d600-4f0b-b60d-b6879ee67a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:31.393508 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e5299-d600-4f0b-b60d-b6879ee67a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:31.394007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e5299-d600-4f0b-b60d-b6879ee67a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:31.394357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e5299-d600-4f0b-b60d-b6879ee67a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:31.395230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e5299-d600-4f0b-b60d-b6879ee67a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:31.396910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e5299-d600-4f0b-b60d-b6879ee67a97 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:31.408030 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 958/1921] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:04:31.413104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effce6b-d455-44de-bd25-7369711f9eda admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:31.424029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effce6b-d455-44de-bd25-7369711f9eda admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:31.429351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a68fe-c1ab-462d-9a20-ee0c32469bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:31.430366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a68fe-c1ab-462d-9a20-ee0c32469bca admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:31.430648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a68fe-c1ab-462d-9a20-ee0c32469bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:31.430932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a68fe-c1ab-462d-9a20-ee0c32469bca admin admin] PATH_INFO: `/v3/projects/e88fd9ccc4e8464fa4120c1a6e15ebee/users/733afed26f8c448e82cc7c697789bc25/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:31.433010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a68fe-c1ab-462d-9a20-ee0c32469bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e88fd9ccc4e8464fa4120c1a6e15ebee, user_id=733afed26f8c448e82cc7c697789bc25, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:31.435760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a68fe-c1ab-462d-9a20-ee0c32469bca admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:31.447748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 964/1922] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:31 2022] PUT /identity/v3/projects/e88fd9ccc4e8464fa4120c1a6e15ebee/users/733afed26f8c448e82cc7c697789bc25/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:31.458969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4713be-09e9-4bcc-b079-47604b35b4a5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:31.459261 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4713be-09e9-4bcc-b079-47604b35b4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:31.459563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4713be-09e9-4bcc-b079-47604b35b4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:31.501922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7a4713be-09e9-4bcc-b079-47604b35b4a5 None None] MFA Rules not processed for user `733afed26f8c448e82cc7c697789bc25`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:31.504591 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7a4713be-09e9-4bcc-b079-47604b35b4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:31.514683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:31.670438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 959/1923] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:31 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:31.682261 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a68fe-c1ab-462d-9a20-ee0c32469bca admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:31.684759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-199a68fe-c1ab-462d-9a20-ee0c32469bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:31.870353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a68fe-c1ab-462d-9a20-ee0c32469bca admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:31.884766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad15f7d-65d5-4de0-a7bb-6f11a14b527e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:31.885893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15f7d-65d5-4de0-a7bb-6f11a14b527e service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:31.886420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15f7d-65d5-4de0-a7bb-6f11a14b527e service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:31.887026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15f7d-65d5-4de0-a7bb-6f11a14b527e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:31.939084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad15f7d-65d5-4de0-a7bb-6f11a14b527e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:31.940608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad15f7d-65d5-4de0-a7bb-6f11a14b527e service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:32.013138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 965/1924] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:32.096359 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4713be-09e9-4bcc-b079-47604b35b4a5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:32.098413 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7a4713be-09e9-4bcc-b079-47604b35b4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:32.284811 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4713be-09e9-4bcc-b079-47604b35b4a5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:32.292040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:32.293467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:32.294042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:32.294525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] PATH_INFO: `/v3/users/1fff94ed9c6946b9a2f6c138c8b6dee4` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:32.296131 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fff94ed9c6946b9a2f6c138c8b6dee4)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:32.298543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:32.384069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fff94ed9c6946b9a2f6c138c8b6dee4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:32.397077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fff94ed9c6946b9a2f6c138c8b6dee4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:32.407409 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fff94ed9c6946b9a2f6c138c8b6dee4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:32.408504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fff94ed9c6946b9a2f6c138c8b6dee4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:32.428164 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fff94ed9c6946b9a2f6c138c8b6dee4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:32.437504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 960/1925] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:32 2022] DELETE /identity/v3/users/1fff94ed9c6946b9a2f6c138c8b6dee4 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:32.443740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230fed-5e43-486f-ab2b-7542a539cdb9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:32.443740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230fed-5e43-486f-ab2b-7542a539cdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:32.443740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230fed-5e43-486f-ab2b-7542a539cdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:32.464498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bd230fed-5e43-486f-ab2b-7542a539cdb9 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:32.465751 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bd230fed-5e43-486f-ab2b-7542a539cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:32.530345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:32.579037 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 966/1926] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:32.596570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:32.598326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:32.630861 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:32.686058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a4942-aa05-4b44-a7a2-5974db5f2d4c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:32.693819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b0d8b-7cec-4a7e-9445-f6b76e01608b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:32.695001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b0d8b-7cec-4a7e-9445-f6b76e01608b service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:32.695504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b0d8b-7cec-4a7e-9445-f6b76e01608b service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:32.696488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b0d8b-7cec-4a7e-9445-f6b76e01608b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:32.713962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b0d8b-7cec-4a7e-9445-f6b76e01608b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:32.716594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b0d8b-7cec-4a7e-9445-f6b76e01608b service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:32.735286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 961/1927] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:04:33.385555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd230fed-5e43-486f-ab2b-7542a539cdb9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:33.397105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bd230fed-5e43-486f-ab2b-7542a539cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:33.419825 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd230fed-5e43-486f-ab2b-7542a539cdb9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:33.449966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55421cdc-1dfb-449f-9058-883453f93338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:33.450761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55421cdc-1dfb-449f-9058-883453f93338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:33.451081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55421cdc-1dfb-449f-9058-883453f93338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:33.451365 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55421cdc-1dfb-449f-9058-883453f93338 admin admin] PATH_INFO: `/v3/projects/f35224175d174ef7996a7ad382015018` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:33.457090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55421cdc-1dfb-449f-9058-883453f93338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f35224175d174ef7996a7ad382015018)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:33.459146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55421cdc-1dfb-449f-9058-883453f93338 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:33.571674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-55421cdc-1dfb-449f-9058-883453f93338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f35224175d174ef7996a7ad382015018', 'request_id': 'req-55421cdc-1dfb-449f-9058-883453f93338'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:33.585887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-55421cdc-1dfb-449f-9058-883453f93338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f35224175d174ef7996a7ad382015018', 'request_id': 'req-55421cdc-1dfb-449f-9058-883453f93338'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:33.607326 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-55421cdc-1dfb-449f-9058-883453f93338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f35224175d174ef7996a7ad382015018', 'request_id': 'req-55421cdc-1dfb-449f-9058-883453f93338'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:33.626282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-55421cdc-1dfb-449f-9058-883453f93338 admin admin] The token cache is being invalidate because project f35224175d174ef7996a7ad382015018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:33.626645 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-55421cdc-1dfb-449f-9058-883453f93338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:33.633076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-55421cdc-1dfb-449f-9058-883453f93338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:33.635030 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 967/1928] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:33 2022] DELETE /identity/v3/projects/f35224175d174ef7996a7ad382015018 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:34.586648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b0d8b-7cec-4a7e-9445-f6b76e01608b service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:34.588268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-147b0d8b-7cec-4a7e-9445-f6b76e01608b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:34.601883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b0d8b-7cec-4a7e-9445-f6b76e01608b service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:34.704913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:34.706109 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:34.706555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:34.706835 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] PATH_INFO: `/v3/users/5e6a87bdb4e949288963524b663edd9c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:34.708485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e6a87bdb4e949288963524b663edd9c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:34.710920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:34.808701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e6a87bdb4e949288963524b663edd9c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:34.820448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e6a87bdb4e949288963524b663edd9c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:34.843338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e6a87bdb4e949288963524b663edd9c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:34.844870 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e6a87bdb4e949288963524b663edd9c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:34.865394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e6a87bdb4e949288963524b663edd9c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:34.889116 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 962/1929] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:34 2022] DELETE /identity/v3/users/5e6a87bdb4e949288963524b663edd9c => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:34.897511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183bb75-f137-4546-9d2d-a782cb098515 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:34.898471 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183bb75-f137-4546-9d2d-a782cb098515 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:34.898713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183bb75-f137-4546-9d2d-a782cb098515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:34.941414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1183bb75-f137-4546-9d2d-a782cb098515 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:34.943363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1183bb75-f137-4546-9d2d-a782cb098515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:35.076999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 968/1930] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:35.130389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:35.137585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:35.241311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:35.275614 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1de16b-decb-4948-bea7-e3110d69b9ec admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:35.296039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf4579-7066-4fcb-a87f-64467563205a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:35.297078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf4579-7066-4fcb-a87f-64467563205a service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:35.297470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf4579-7066-4fcb-a87f-64467563205a service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:35.297827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf4579-7066-4fcb-a87f-64467563205a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:35.330219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf4579-7066-4fcb-a87f-64467563205a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:35.331202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf4579-7066-4fcb-a87f-64467563205a service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:35.364050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 963/1931] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:04:36.218947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183bb75-f137-4546-9d2d-a782cb098515 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:36.221397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1183bb75-f137-4546-9d2d-a782cb098515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:36.242406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183bb75-f137-4546-9d2d-a782cb098515 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:36.256853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:36.257812 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:36.258259 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:36.258537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] PATH_INFO: `/v3/projects/4a95d33c4ef74901bf320207aaf64fb0` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:36.259938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a95d33c4ef74901bf320207aaf64fb0)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:36.261970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:36.335751 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a95d33c4ef74901bf320207aaf64fb0', 'request_id': 'req-9564f449-4e35-4ad4-9836-fc1395038c77'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:36.342847 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:36.344812 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a95d33c4ef74901bf320207aaf64fb0', 'request_id': 'req-9564f449-4e35-4ad4-9836-fc1395038c77'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:36.354112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a95d33c4ef74901bf320207aaf64fb0', 'request_id': 'req-9564f449-4e35-4ad4-9836-fc1395038c77'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:36.376326 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] The token cache is being invalidate because project 4a95d33c4ef74901bf320207aaf64fb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:36.376761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:36.377910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:36.386232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 969/1932] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:36 2022] DELETE /identity/v3/projects/4a95d33c4ef74901bf320207aaf64fb0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:36.387673 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf4579-7066-4fcb-a87f-64467563205a service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:36.417890 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf4579-7066-4fcb-a87f-64467563205a service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:36.569317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b4c2c-3c7f-4500-9c82-25f4d8d8c4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:36.570815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4c2c-3c7f-4500-9c82-25f4d8d8c4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:36.570815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4c2c-3c7f-4500-9c82-25f4d8d8c4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:36.571135 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4c2c-3c7f-4500-9c82-25f4d8d8c4ab admin admin] PATH_INFO: `/v3/users/733afed26f8c448e82cc7c697789bc25` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:36.572790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b4c2c-3c7f-4500-9c82-25f4d8d8c4ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=733afed26f8c448e82cc7c697789bc25)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:36.575077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b4c2c-3c7f-4500-9c82-25f4d8d8c4ab admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:36.689967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-701b4c2c-3c7f-4500-9c82-25f4d8d8c4ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '733afed26f8c448e82cc7c697789bc25'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:36.710749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-701b4c2c-3c7f-4500-9c82-25f4d8d8c4ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '733afed26f8c448e82cc7c697789bc25'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:36.731691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-701b4c2c-3c7f-4500-9c82-25f4d8d8c4ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '733afed26f8c448e82cc7c697789bc25'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:36.733493 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-701b4c2c-3c7f-4500-9c82-25f4d8d8c4ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '733afed26f8c448e82cc7c697789bc25'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:36.777427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-701b4c2c-3c7f-4500-9c82-25f4d8d8c4ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '733afed26f8c448e82cc7c697789bc25'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:36.802828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 964/1933] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:36 2022] DELETE /identity/v3/users/733afed26f8c448e82cc7c697789bc25 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:37.450362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:37.453936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:37.493560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564f449-4e35-4ad4-9836-fc1395038c77 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:37.594074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d65a4-99c0-4729-8e20-e60f4a4115ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:37.595418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d65a4-99c0-4729-8e20-e60f4a4115ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:37.595933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d65a4-99c0-4729-8e20-e60f4a4115ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:37.596198 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d65a4-99c0-4729-8e20-e60f4a4115ef admin admin] PATH_INFO: `/v3/projects/e88fd9ccc4e8464fa4120c1a6e15ebee` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:37.601235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d65a4-99c0-4729-8e20-e60f4a4115ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=e88fd9ccc4e8464fa4120c1a6e15ebee)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:37.603974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d65a4-99c0-4729-8e20-e60f4a4115ef admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:37.687019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be3d65a4-99c0-4729-8e20-e60f4a4115ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e88fd9ccc4e8464fa4120c1a6e15ebee', 'request_id': 'req-be3d65a4-99c0-4729-8e20-e60f4a4115ef'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:37.726494 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be3d65a4-99c0-4729-8e20-e60f4a4115ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e88fd9ccc4e8464fa4120c1a6e15ebee', 'request_id': 'req-be3d65a4-99c0-4729-8e20-e60f4a4115ef'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:37.750350 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be3d65a4-99c0-4729-8e20-e60f4a4115ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e88fd9ccc4e8464fa4120c1a6e15ebee', 'request_id': 'req-be3d65a4-99c0-4729-8e20-e60f4a4115ef'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:37.788070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be3d65a4-99c0-4729-8e20-e60f4a4115ef admin admin] The token cache is being invalidate because project e88fd9ccc4e8464fa4120c1a6e15ebee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:37.791061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be3d65a4-99c0-4729-8e20-e60f4a4115ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:37.792038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be3d65a4-99c0-4729-8e20-e60f4a4115ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:37.794103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 970/1934] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:37 2022] DELETE /identity/v3/projects/e88fd9ccc4e8464fa4120c1a6e15ebee => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:37.801246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bde0c-f5e1-4a5c-8f81-736fae302c0c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:37.801737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bde0c-f5e1-4a5c-8f81-736fae302c0c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:37.802001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bde0c-f5e1-4a5c-8f81-736fae302c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:37.835094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9e0bde0c-f5e1-4a5c-8f81-736fae302c0c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:37.837325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9e0bde0c-f5e1-4a5c-8f81-736fae302c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:37.986363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 965/1935] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:37.996361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1e4c4-a26f-4dc1-b18b-31b81f029a1c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:37.996766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1e4c4-a26f-4dc1-b18b-31b81f029a1c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:37.997058 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1e4c4-a26f-4dc1-b18b-31b81f029a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:38.022488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-47c1e4c4-a26f-4dc1-b18b-31b81f029a1c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:38.023734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-47c1e4c4-a26f-4dc1-b18b-31b81f029a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:38.033392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 971/1936] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:38.038193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bde0c-f5e1-4a5c-8f81-736fae302c0c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:38.050095 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bde0c-f5e1-4a5c-8f81-736fae302c0c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:38.057153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a5689-07e9-4782-b536-89ff2853af00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:38.057895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a5689-07e9-4782-b536-89ff2853af00 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:38.058185 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a5689-07e9-4782-b536-89ff2853af00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:38.058467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a5689-07e9-4782-b536-89ff2853af00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:38.059498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a5689-07e9-4782-b536-89ff2853af00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:04:38.059814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a5689-07e9-4782-b536-89ff2853af00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:38.061654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a5689-07e9-4782-b536-89ff2853af00 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:38.064781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:38.091343 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 966/1937] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:04:38.092482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1e4c4-a26f-4dc1-b18b-31b81f029a1c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:38.104617 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1e4c4-a26f-4dc1-b18b-31b81f029a1c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:38.122532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f96d08-ced7-48b5-92be-08e9f3ef0267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:38.123578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f96d08-ced7-48b5-92be-08e9f3ef0267 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:38.123916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f96d08-ced7-48b5-92be-08e9f3ef0267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:38.124191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f96d08-ced7-48b5-92be-08e9f3ef0267 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:38.124985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f96d08-ced7-48b5-92be-08e9f3ef0267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:38.127597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f96d08-ced7-48b5-92be-08e9f3ef0267 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:38.232023 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a5689-07e9-4782-b536-89ff2853af00 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:38.233855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 972/1938] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:38 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:38.247718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a5689-07e9-4782-b536-89ff2853af00 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:38.407683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b30dd-2b1b-41db-b93f-c395abe63d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:38.408451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b30dd-2b1b-41db-b93f-c395abe63d07 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:38.408735 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b30dd-2b1b-41db-b93f-c395abe63d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:38.409012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b30dd-2b1b-41db-b93f-c395abe63d07 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:38.409837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b30dd-2b1b-41db-b93f-c395abe63d07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:38.411999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b30dd-2b1b-41db-b93f-c395abe63d07 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:38.452718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 967/1939] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:38 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:04:38.456002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f96d08-ced7-48b5-92be-08e9f3ef0267 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:38.506587 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f96d08-ced7-48b5-92be-08e9f3ef0267 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:38.535657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43162d55-305d-4945-aced-24589018f9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:38.538263 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43162d55-305d-4945-aced-24589018f9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:38.538731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43162d55-305d-4945-aced-24589018f9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:38.539167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43162d55-305d-4945-aced-24589018f9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:38.540581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43162d55-305d-4945-aced-24589018f9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:38.556698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43162d55-305d-4945-aced-24589018f9a2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:38.621660 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b30dd-2b1b-41db-b93f-c395abe63d07 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:38.625064 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 973/1940] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:38.654198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b30dd-2b1b-41db-b93f-c395abe63d07 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:38.660040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a5333-192f-42b6-86d9-24b17df47dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:38.661228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a5333-192f-42b6-86d9-24b17df47dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:38.661797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a5333-192f-42b6-86d9-24b17df47dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:38.662436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a5333-192f-42b6-86d9-24b17df47dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:38.664110 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a5333-192f-42b6-86d9-24b17df47dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:38.667710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a5333-192f-42b6-86d9-24b17df47dcd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:38.704738 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43162d55-305d-4945-aced-24589018f9a2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:38.713828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 968/1941] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:38.728092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43162d55-305d-4945-aced-24589018f9a2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:38.738029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8a496-8670-4c93-9f20-a6e014c6b66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:38.741534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8a496-8670-4c93-9f20-a6e014c6b66e admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:38.742537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8a496-8670-4c93-9f20-a6e014c6b66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:38.743778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8a496-8670-4c93-9f20-a6e014c6b66e admin admin] PATH_INFO: `/v3/projects/187ad982fc4548369ac093ca9d48acab/users/9fefd5e007b44a08b7f838ee96168a1f/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:38.782833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8a496-8670-4c93-9f20-a6e014c6b66e admin admin] RBAC: Authorizing `identity:create_grant(project_id=187ad982fc4548369ac093ca9d48acab, user_id=9fefd5e007b44a08b7f838ee96168a1f, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:38.797824 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8a496-8670-4c93-9f20-a6e014c6b66e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:38.850658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 974/1942] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:38 2022] PUT /identity/v3/projects/187ad982fc4548369ac093ca9d48acab/users/9fefd5e007b44a08b7f838ee96168a1f/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:38.868448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe02a04-d702-4d37-83e8-470a46a52d56 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:38.869319 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe02a04-d702-4d37-83e8-470a46a52d56 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:38.869319 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe02a04-d702-4d37-83e8-470a46a52d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:38.954589 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1fe02a04-d702-4d37-83e8-470a46a52d56 None None] MFA Rules not processed for user `9fefd5e007b44a08b7f838ee96168a1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:38.956531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1fe02a04-d702-4d37-83e8-470a46a52d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:39.235596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 969/1943] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:39.237481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8a496-8670-4c93-9f20-a6e014c6b66e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:39.264966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:39.367467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8a496-8670-4c93-9f20-a6e014c6b66e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:39.376500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb1caa-8530-461a-ae2c-de1fe9c8c045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:39.378119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb1caa-8530-461a-ae2c-de1fe9c8c045 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:39.378700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb1caa-8530-461a-ae2c-de1fe9c8c045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:39.379256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb1caa-8530-461a-ae2c-de1fe9c8c045 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:39.380673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb1caa-8530-461a-ae2c-de1fe9c8c045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:39.385024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb1caa-8530-461a-ae2c-de1fe9c8c045 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:39.440338 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 975/1944] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:39 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:39.442586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe02a04-d702-4d37-83e8-470a46a52d56 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:39.461719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe02a04-d702-4d37-83e8-470a46a52d56 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:39.546856 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e3268-a1a7-49fa-9d0b-08dbd11db5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:39.547609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e3268-a1a7-49fa-9d0b-08dbd11db5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:39.548150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e3268-a1a7-49fa-9d0b-08dbd11db5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:39.548448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e3268-a1a7-49fa-9d0b-08dbd11db5c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:39.549170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e3268-a1a7-49fa-9d0b-08dbd11db5c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:39.551726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e3268-a1a7-49fa-9d0b-08dbd11db5c4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:39.562555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:39.605724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 970/1945] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:39 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:39.609606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb1caa-8530-461a-ae2c-de1fe9c8c045 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:39.625315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb1caa-8530-461a-ae2c-de1fe9c8c045 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:39.632425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdfcce-2985-4f6d-8e78-4f67bd6f2497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:39.633498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdfcce-2985-4f6d-8e78-4f67bd6f2497 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:39.633991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdfcce-2985-4f6d-8e78-4f67bd6f2497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:39.634265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdfcce-2985-4f6d-8e78-4f67bd6f2497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:39.635252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacdfcce-2985-4f6d-8e78-4f67bd6f2497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:39.637127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacdfcce-2985-4f6d-8e78-4f67bd6f2497 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:39.652970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 976/1946] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:04:39.657274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e3268-a1a7-49fa-9d0b-08dbd11db5c4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:39.677215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e3268-a1a7-49fa-9d0b-08dbd11db5c4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:39.697293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ef1d3-8697-456b-8d54-13c6a6804f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:39.698237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ef1d3-8697-456b-8d54-13c6a6804f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:39.698585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ef1d3-8697-456b-8d54-13c6a6804f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:39.698916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ef1d3-8697-456b-8d54-13c6a6804f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:39.700232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ef1d3-8697-456b-8d54-13c6a6804f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:39.703323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ef1d3-8697-456b-8d54-13c6a6804f00 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:39.721470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 971/1947] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:39.726383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdfcce-2985-4f6d-8e78-4f67bd6f2497 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:39.739470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdfcce-2985-4f6d-8e78-4f67bd6f2497 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:39.760550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcf5f7-9dba-47b0-89a9-e1d7a5f8cfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:39.762005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcf5f7-9dba-47b0-89a9-e1d7a5f8cfba admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:39.762503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcf5f7-9dba-47b0-89a9-e1d7a5f8cfba admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:39.763010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcf5f7-9dba-47b0-89a9-e1d7a5f8cfba admin admin] PATH_INFO: `/v3/projects/9cf3bb45c9484788983a2e0b670b8b0e/users/4effd323481d43cba5ce43d7ac9ba9cb/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:39.826839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcf5f7-9dba-47b0-89a9-e1d7a5f8cfba admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf3bb45c9484788983a2e0b670b8b0e, user_id=4effd323481d43cba5ce43d7ac9ba9cb, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:39.829957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcf5f7-9dba-47b0-89a9-e1d7a5f8cfba admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:39.857683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 977/1948] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:39 2022] PUT /identity/v3/projects/9cf3bb45c9484788983a2e0b670b8b0e/users/4effd323481d43cba5ce43d7ac9ba9cb/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:39.864703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a97e59-6da5-4020-bc3a-71dbc42f7974 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:39.865021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a97e59-6da5-4020-bc3a-71dbc42f7974 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:39.865300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a97e59-6da5-4020-bc3a-71dbc42f7974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:39.916535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-50a97e59-6da5-4020-bc3a-71dbc42f7974 None None] MFA Rules not processed for user `4effd323481d43cba5ce43d7ac9ba9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:39.918731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-50a97e59-6da5-4020-bc3a-71dbc42f7974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:40.037284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 972/1949] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:40.041212 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcf5f7-9dba-47b0-89a9-e1d7a5f8cfba admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:40.168183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcf5f7-9dba-47b0-89a9-e1d7a5f8cfba admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:40.178284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5af2ea-0998-46ef-abcf-64b60b8837c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:40.179462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5af2ea-0998-46ef-abcf-64b60b8837c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:40.179958 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5af2ea-0998-46ef-abcf-64b60b8837c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:40.180448 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5af2ea-0998-46ef-abcf-64b60b8837c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:40.182162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5af2ea-0998-46ef-abcf-64b60b8837c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:40.186389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5af2ea-0998-46ef-abcf-64b60b8837c8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:40.235371 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 978/1950] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:40 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:40.241154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a97e59-6da5-4020-bc3a-71dbc42f7974 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:40.259185 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a97e59-6da5-4020-bc3a-71dbc42f7974 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:40.358582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1f57c-ac29-424e-83a8-b2df9893038f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:40.359640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1f57c-ac29-424e-83a8-b2df9893038f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:40.360099 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1f57c-ac29-424e-83a8-b2df9893038f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:40.360438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1f57c-ac29-424e-83a8-b2df9893038f admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:40.361237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1f57c-ac29-424e-83a8-b2df9893038f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:40.364156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1f57c-ac29-424e-83a8-b2df9893038f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:40.406303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5af2ea-0998-46ef-abcf-64b60b8837c8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:40.406966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 973/1951] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:40 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:04:40.425420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5af2ea-0998-46ef-abcf-64b60b8837c8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:40.434618 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1fad8-893d-44ac-8ad7-5e31b83c59bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:40.435763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1fad8-893d-44ac-8ad7-5e31b83c59bb admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:40.436269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1fad8-893d-44ac-8ad7-5e31b83c59bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:40.436766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1fad8-893d-44ac-8ad7-5e31b83c59bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:40.438850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1fad8-893d-44ac-8ad7-5e31b83c59bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:40.442216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1fad8-893d-44ac-8ad7-5e31b83c59bb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:40.475524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 979/1952] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:40.476441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1f57c-ac29-424e-83a8-b2df9893038f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:40.500636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1f57c-ac29-424e-83a8-b2df9893038f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:40.514353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59652a4c-2e7b-42ac-8c15-7072997579b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:40.515929 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59652a4c-2e7b-42ac-8c15-7072997579b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:40.516417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59652a4c-2e7b-42ac-8c15-7072997579b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:40.516903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59652a4c-2e7b-42ac-8c15-7072997579b7 admin admin] PATH_INFO: `/v3/projects/b433490dc30e49ef8dc2b7d13416f5e0/users/e36477d11dba4c8ea529bfea64a5e9c2/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:40.546985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59652a4c-2e7b-42ac-8c15-7072997579b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b433490dc30e49ef8dc2b7d13416f5e0, user_id=e36477d11dba4c8ea529bfea64a5e9c2, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:40.550421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59652a4c-2e7b-42ac-8c15-7072997579b7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:40.572056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 974/1953] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:40 2022] PUT /identity/v3/projects/b433490dc30e49ef8dc2b7d13416f5e0/users/e36477d11dba4c8ea529bfea64a5e9c2/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:40.578295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1fad8-893d-44ac-8ad7-5e31b83c59bb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:40.608479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1fad8-893d-44ac-8ad7-5e31b83c59bb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:40.668744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:40.733176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acbf21-164f-4417-afef-984f00fa7d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:40.734067 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acbf21-164f-4417-afef-984f00fa7d8f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:40.734605 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acbf21-164f-4417-afef-984f00fa7d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:40.734940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acbf21-164f-4417-afef-984f00fa7d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:40.735783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acbf21-164f-4417-afef-984f00fa7d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:40.737650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acbf21-164f-4417-afef-984f00fa7d8f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:40.756919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 980/1954] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:40.772501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59652a4c-2e7b-42ac-8c15-7072997579b7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:40.795212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59652a4c-2e7b-42ac-8c15-7072997579b7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:40.806212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fae902-075b-4942-b230-cec5da0d5213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:40.809527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fae902-075b-4942-b230-cec5da0d5213 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:40.810268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fae902-075b-4942-b230-cec5da0d5213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:40.810873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fae902-075b-4942-b230-cec5da0d5213 admin admin] PATH_INFO: `/v3/projects/b433490dc30e49ef8dc2b7d13416f5e0/users/e36477d11dba4c8ea529bfea64a5e9c2/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:40.817594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fae902-075b-4942-b230-cec5da0d5213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b433490dc30e49ef8dc2b7d13416f5e0, user_id=e36477d11dba4c8ea529bfea64a5e9c2, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:40.824623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fae902-075b-4942-b230-cec5da0d5213 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:40.844982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 975/1955] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:40 2022] PUT /identity/v3/projects/b433490dc30e49ef8dc2b7d13416f5e0/users/e36477d11dba4c8ea529bfea64a5e9c2/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:40.856696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de8179-07ee-4117-b05d-445ec3b9181a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:40.857110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de8179-07ee-4117-b05d-445ec3b9181a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:40.857268 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de8179-07ee-4117-b05d-445ec3b9181a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:40.895207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d9de8179-07ee-4117-b05d-445ec3b9181a None None] MFA Rules not processed for user `e36477d11dba4c8ea529bfea64a5e9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:40.896564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d9de8179-07ee-4117-b05d-445ec3b9181a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:41.027843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 981/1956] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:41.095949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fae902-075b-4942-b230-cec5da0d5213 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:41.098512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e7fae902-075b-4942-b230-cec5da0d5213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:41.136495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:41.182831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fae902-075b-4942-b230-cec5da0d5213 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:41.188746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10472e86-62fe-4ff4-a27d-0842034255d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:41.188746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10472e86-62fe-4ff4-a27d-0842034255d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:41.188746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10472e86-62fe-4ff4-a27d-0842034255d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:41.188746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10472e86-62fe-4ff4-a27d-0842034255d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:41.204479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10472e86-62fe-4ff4-a27d-0842034255d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:41.205344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10472e86-62fe-4ff4-a27d-0842034255d9 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:41.244779 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 976/1957] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:42.019728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de8179-07ee-4117-b05d-445ec3b9181a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:42.021298 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d9de8179-07ee-4117-b05d-445ec3b9181a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:42.036225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de8179-07ee-4117-b05d-445ec3b9181a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:42.046881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24224a25-1cae-4e3e-b558-df052c9b574b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:42.047574 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24224a25-1cae-4e3e-b558-df052c9b574b service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:42.047868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24224a25-1cae-4e3e-b558-df052c9b574b service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:42.048149 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24224a25-1cae-4e3e-b558-df052c9b574b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:42.129683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24224a25-1cae-4e3e-b558-df052c9b574b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:42.131391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24224a25-1cae-4e3e-b558-df052c9b574b service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:42.147358 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 982/1958] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:49.590712 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10472e86-62fe-4ff4-a27d-0842034255d9 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:49.600631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10472e86-62fe-4ff4-a27d-0842034255d9 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:49.604490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9356f76-e918-4182-af37-98c4dbc2ce74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:49.605153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9356f76-e918-4182-af37-98c4dbc2ce74 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:49.605437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9356f76-e918-4182-af37-98c4dbc2ce74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:49.605758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9356f76-e918-4182-af37-98c4dbc2ce74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:49.685183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9356f76-e918-4182-af37-98c4dbc2ce74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:49.686519 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9356f76-e918-4182-af37-98c4dbc2ce74 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:49.698264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 977/1959] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:52.137249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=75 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:52.148468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24224a25-1cae-4e3e-b558-df052c9b574b service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:52.237468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24224a25-1cae-4e3e-b558-df052c9b574b service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:52.242830 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:52.243730 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:52.244014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:52.244298 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] PATH_INFO: `/v3/users/9fefd5e007b44a08b7f838ee96168a1f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:52.245594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fefd5e007b44a08b7f838ee96168a1f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:52.247611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:52.311152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fefd5e007b44a08b7f838ee96168a1f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:52.311995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fefd5e007b44a08b7f838ee96168a1f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:52.312603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fefd5e007b44a08b7f838ee96168a1f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:52.322268 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fefd5e007b44a08b7f838ee96168a1f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:52.338418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fefd5e007b44a08b7f838ee96168a1f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:52.355322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 983/1960] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:52 2022] DELETE /identity/v3/users/9fefd5e007b44a08b7f838ee96168a1f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:52.360499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328de56f-0d13-42d9-be40-f9897123440c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:52.361505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328de56f-0d13-42d9-be40-f9897123440c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:52.361884 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328de56f-0d13-42d9-be40-f9897123440c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:52.384662 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-328de56f-0d13-42d9-be40-f9897123440c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:52.386030 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-328de56f-0d13-42d9-be40-f9897123440c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:52.461480 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 978/1961] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:52.472395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:52.474757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:52.548847 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f77774-26d0-4cb6-bb94-712681366feb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:52.553582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7952b-d65b-4d9a-a2a4-db1bdf83350c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:52.554397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7952b-d65b-4d9a-a2a4-db1bdf83350c service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:52.554716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7952b-d65b-4d9a-a2a4-db1bdf83350c service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:52.555076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7952b-d65b-4d9a-a2a4-db1bdf83350c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:52.566350 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c7952b-d65b-4d9a-a2a4-db1bdf83350c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:52.567890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c7952b-d65b-4d9a-a2a4-db1bdf83350c service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:52.580569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 984/1962] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:52.971850 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328de56f-0d13-42d9-be40-f9897123440c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:52.973593 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-328de56f-0d13-42d9-be40-f9897123440c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:52.987314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328de56f-0d13-42d9-be40-f9897123440c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:52.995043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:52.995637 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:52.995924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:52.996205 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] PATH_INFO: `/v3/projects/187ad982fc4548369ac093ca9d48acab` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:52.997582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=187ad982fc4548369ac093ca9d48acab)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:53.000204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:53.066236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '187ad982fc4548369ac093ca9d48acab', 'request_id': 'req-f18cc768-3a99-42da-94b8-86248aa43fde'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.073717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '187ad982fc4548369ac093ca9d48acab', 'request_id': 'req-f18cc768-3a99-42da-94b8-86248aa43fde'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.075907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:53.085381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '187ad982fc4548369ac093ca9d48acab', 'request_id': 'req-f18cc768-3a99-42da-94b8-86248aa43fde'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.101720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] The token cache is being invalidate because project 187ad982fc4548369ac093ca9d48acab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:53.102102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.102826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.105982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 979/1963] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:52 2022] DELETE /identity/v3/projects/187ad982fc4548369ac093ca9d48acab => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:53.111338 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7952b-d65b-4d9a-a2a4-db1bdf83350c service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:53.128629 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7952b-d65b-4d9a-a2a4-db1bdf83350c service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:53.165914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:53.167417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:53.168925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:53.228634 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5b9ea-53ff-4a54-9de2-20f305b7be2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:53.229708 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5b9ea-53ff-4a54-9de2-20f305b7be2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:53.230023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5b9ea-53ff-4a54-9de2-20f305b7be2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:53.230401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5b9ea-53ff-4a54-9de2-20f305b7be2c admin admin] PATH_INFO: `/v3/users/4effd323481d43cba5ce43d7ac9ba9cb` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:53.231842 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5b9ea-53ff-4a54-9de2-20f305b7be2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4effd323481d43cba5ce43d7ac9ba9cb)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:53.233995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5b9ea-53ff-4a54-9de2-20f305b7be2c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:53.299298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cc768-3a99-42da-94b8-86248aa43fde admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:53.308514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:53.310088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:53.310586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:53.311097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] PATH_INFO: `/v3/users/87d8f1a5a0324ca893ddaab1905023d4` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:53.311225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ada5b9ea-53ff-4a54-9de2-20f305b7be2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4effd323481d43cba5ce43d7ac9ba9cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.311840 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ada5b9ea-53ff-4a54-9de2-20f305b7be2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4effd323481d43cba5ce43d7ac9ba9cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.312516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ada5b9ea-53ff-4a54-9de2-20f305b7be2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4effd323481d43cba5ce43d7ac9ba9cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.313530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] RBAC: Authorizing `identity:delete_user(user_id=87d8f1a5a0324ca893ddaab1905023d4)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:53.316903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:53.322893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ada5b9ea-53ff-4a54-9de2-20f305b7be2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4effd323481d43cba5ce43d7ac9ba9cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.343167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ada5b9ea-53ff-4a54-9de2-20f305b7be2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4effd323481d43cba5ce43d7ac9ba9cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.367870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 985/1964] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:53 2022] DELETE /identity/v3/users/4effd323481d43cba5ce43d7ac9ba9cb => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:53.430833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87d8f1a5a0324ca893ddaab1905023d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.445477 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87d8f1a5a0324ca893ddaab1905023d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.462974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87d8f1a5a0324ca893ddaab1905023d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.464587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87d8f1a5a0324ca893ddaab1905023d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.493044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87d8f1a5a0324ca893ddaab1905023d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:53.505971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 980/1965] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:53 2022] DELETE /identity/v3/users/87d8f1a5a0324ca893ddaab1905023d4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:53.513684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1b8cd-86bb-4588-8a73-3432fa395333 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:53.514283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1b8cd-86bb-4588-8a73-3432fa395333 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:53.514736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1b8cd-86bb-4588-8a73-3432fa395333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:53.544403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-27f1b8cd-86bb-4588-8a73-3432fa395333 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:53.545779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-27f1b8cd-86bb-4588-8a73-3432fa395333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:53.670339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 986/1966] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:53.682105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:53.685689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:53.760478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca8109-6a57-474a-a745-16926171640a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:53.764510 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670d402-a5f0-4017-907f-31fe3e20ee6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:53.765191 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670d402-a5f0-4017-907f-31fe3e20ee6c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:53.765470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670d402-a5f0-4017-907f-31fe3e20ee6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:53.765785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670d402-a5f0-4017-907f-31fe3e20ee6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:53.779961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670d402-a5f0-4017-907f-31fe3e20ee6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:53.781471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670d402-a5f0-4017-907f-31fe3e20ee6c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:53.793108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 981/1967] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:53.989434 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1b8cd-86bb-4588-8a73-3432fa395333 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:53.992231 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-27f1b8cd-86bb-4588-8a73-3432fa395333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:54.027980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1b8cd-86bb-4588-8a73-3432fa395333 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:54.039088 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:54.040936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:54.041591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:54.042352 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] PATH_INFO: `/v3/projects/9cf3bb45c9484788983a2e0b670b8b0e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:54.046884 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cf3bb45c9484788983a2e0b670b8b0e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:54.048981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:54.128880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:54.153463 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cf3bb45c9484788983a2e0b670b8b0e', 'request_id': 'req-c6f931d9-96d5-41d2-a7f0-40dac2956709'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.163555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cf3bb45c9484788983a2e0b670b8b0e', 'request_id': 'req-c6f931d9-96d5-41d2-a7f0-40dac2956709'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.174770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670d402-a5f0-4017-907f-31fe3e20ee6c service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:54.177147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3670d402-a5f0-4017-907f-31fe3e20ee6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:54.182442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cf3bb45c9484788983a2e0b670b8b0e', 'request_id': 'req-c6f931d9-96d5-41d2-a7f0-40dac2956709'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.208881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] The token cache is being invalidate because project 9cf3bb45c9484788983a2e0b670b8b0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:54.209550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.211379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.215359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 987/1968] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:53 2022] DELETE /identity/v3/projects/9cf3bb45c9484788983a2e0b670b8b0e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:54.220244 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670d402-a5f0-4017-907f-31fe3e20ee6c service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:54.222075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:54.225092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:54.229537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:54.327702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fd72b-65e8-43ed-86c6-da299ce88e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:54.329677 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fd72b-65e8-43ed-86c6-da299ce88e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:54.330282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fd72b-65e8-43ed-86c6-da299ce88e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:54.330897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fd72b-65e8-43ed-86c6-da299ce88e78 admin admin] PATH_INFO: `/v3/projects/b8d567be1b3a43d18a9f185416845815` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:54.333785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318fd72b-65e8-43ed-86c6-da299ce88e78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8d567be1b3a43d18a9f185416845815)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:54.337937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318fd72b-65e8-43ed-86c6-da299ce88e78 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:54.355633 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f931d9-96d5-41d2-a7f0-40dac2956709 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:54.372786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:54.374083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:54.374655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:54.375016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] PATH_INFO: `/v3/users/e36477d11dba4c8ea529bfea64a5e9c2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:54.376524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e36477d11dba4c8ea529bfea64a5e9c2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:54.379704 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:54.442036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-318fd72b-65e8-43ed-86c6-da299ce88e78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8d567be1b3a43d18a9f185416845815', 'request_id': 'req-318fd72b-65e8-43ed-86c6-da299ce88e78'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.446758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e36477d11dba4c8ea529bfea64a5e9c2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.447758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e36477d11dba4c8ea529bfea64a5e9c2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.448406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e36477d11dba4c8ea529bfea64a5e9c2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.457140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e36477d11dba4c8ea529bfea64a5e9c2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.457610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-318fd72b-65e8-43ed-86c6-da299ce88e78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8d567be1b3a43d18a9f185416845815', 'request_id': 'req-318fd72b-65e8-43ed-86c6-da299ce88e78'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.470724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-318fd72b-65e8-43ed-86c6-da299ce88e78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8d567be1b3a43d18a9f185416845815', 'request_id': 'req-318fd72b-65e8-43ed-86c6-da299ce88e78'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.473816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e36477d11dba4c8ea529bfea64a5e9c2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.493603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-318fd72b-65e8-43ed-86c6-da299ce88e78 admin admin] The token cache is being invalidate because project b8d567be1b3a43d18a9f185416845815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:54.494274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-318fd72b-65e8-43ed-86c6-da299ce88e78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.495433 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-318fd72b-65e8-43ed-86c6-da299ce88e78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.495863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 988/1969] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:54 2022] DELETE /identity/v3/users/e36477d11dba4c8ea529bfea64a5e9c2 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:54.497546 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:54.500005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:54.507914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 982/1970] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:54 2022] DELETE /identity/v3/projects/b8d567be1b3a43d18a9f185416845815 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:54.516672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fe785-e04b-4cc9-b9dc-47229bd99d37 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:54.517341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fe785-e04b-4cc9-b9dc-47229bd99d37 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:54.517945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fe785-e04b-4cc9-b9dc-47229bd99d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:54.548464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-eb6fe785-e04b-4cc9-b9dc-47229bd99d37 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:54.550349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-eb6fe785-e04b-4cc9-b9dc-47229bd99d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:54.588470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:54.625702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c584e3c-4060-48fe-a0c3-960c565a51fb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:54.640646 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d510f-a1ee-4c5f-8fc2-77a47e61392f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:54.641597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d510f-a1ee-4c5f-8fc2-77a47e61392f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:54.642015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d510f-a1ee-4c5f-8fc2-77a47e61392f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:54.642240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d510f-a1ee-4c5f-8fc2-77a47e61392f admin admin] PATH_INFO: `/v3/users/2b48cf33019b4e98a7ac34b66e7e5652` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:54.645736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d510f-a1ee-4c5f-8fc2-77a47e61392f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b48cf33019b4e98a7ac34b66e7e5652)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:54.647864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d510f-a1ee-4c5f-8fc2-77a47e61392f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:54.704311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 983/1971] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:54.707307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27d070-4f75-4908-b212-04e12bec6cbd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:54.707616 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27d070-4f75-4908-b212-04e12bec6cbd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:54.707886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27d070-4f75-4908-b212-04e12bec6cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:54.729721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cb27d070-4f75-4908-b212-04e12bec6cbd None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:54.731185 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cb27d070-4f75-4908-b212-04e12bec6cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:54.740227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 984/1972] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:54.747579 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-068d510f-a1ee-4c5f-8fc2-77a47e61392f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b48cf33019b4e98a7ac34b66e7e5652'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.747997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27d070-4f75-4908-b212-04e12bec6cbd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:54.749128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-068d510f-a1ee-4c5f-8fc2-77a47e61392f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b48cf33019b4e98a7ac34b66e7e5652'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.749872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-068d510f-a1ee-4c5f-8fc2-77a47e61392f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b48cf33019b4e98a7ac34b66e7e5652'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.761498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27d070-4f75-4908-b212-04e12bec6cbd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:54.762376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-068d510f-a1ee-4c5f-8fc2-77a47e61392f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b48cf33019b4e98a7ac34b66e7e5652'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.769141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0377ca6-c1b2-4e25-b3fe-7b2bd90ff5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:54.770851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0377ca6-c1b2-4e25-b3fe-7b2bd90ff5ab admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:54.771140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0377ca6-c1b2-4e25-b3fe-7b2bd90ff5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:54.771433 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0377ca6-c1b2-4e25-b3fe-7b2bd90ff5ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:54.772351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0377ca6-c1b2-4e25-b3fe-7b2bd90ff5ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:04:54.772702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0377ca6-c1b2-4e25-b3fe-7b2bd90ff5ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:54.774881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0377ca6-c1b2-4e25-b3fe-7b2bd90ff5ab admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:54.782024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-068d510f-a1ee-4c5f-8fc2-77a47e61392f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b48cf33019b4e98a7ac34b66e7e5652'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:54.801206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 985/1973] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:04:54.802509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0377ca6-c1b2-4e25-b3fe-7b2bd90ff5ab admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:54.806969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f0377ca6-c1b2-4e25-b3fe-7b2bd90ff5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:54.814516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 989/1974] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:54 2022] DELETE /identity/v3/users/2b48cf33019b4e98a7ac34b66e7e5652 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:54.820292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f645b-41fb-4d30-b944-5586a1ca8b45 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:54.820623 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f645b-41fb-4d30-b944-5586a1ca8b45 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:54.821056 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f645b-41fb-4d30-b944-5586a1ca8b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:54.832816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0377ca6-c1b2-4e25-b3fe-7b2bd90ff5ab admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:54.849912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-614f645b-41fb-4d30-b944-5586a1ca8b45 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:54.852029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-614f645b-41fb-4d30-b944-5586a1ca8b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:54.862229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:54.930153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc676246-6d9b-408e-8551-7fae22a99ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:54.930915 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc676246-6d9b-408e-8551-7fae22a99ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:54.931200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc676246-6d9b-408e-8551-7fae22a99ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:54.931493 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc676246-6d9b-408e-8551-7fae22a99ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:54.932215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc676246-6d9b-408e-8551-7fae22a99ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:54.934347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc676246-6d9b-408e-8551-7fae22a99ef7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:54.970071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 986/1975] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:54 2022] POST /identity/v3/projects => generated 365 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:54.974608 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc676246-6d9b-408e-8551-7fae22a99ef7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:54.979842 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 990/1976] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:54.987982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f645b-41fb-4d30-b944-5586a1ca8b45 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:54.989518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-614f645b-41fb-4d30-b944-5586a1ca8b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:55.079563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f645b-41fb-4d30-b944-5586a1ca8b45 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:55.089818 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc676246-6d9b-408e-8551-7fae22a99ef7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:55.096909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba72497-1ca0-4bdf-bcb8-8797d9dd759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:55.097655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba72497-1ca0-4bdf-bcb8-8797d9dd759f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.097978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba72497-1ca0-4bdf-bcb8-8797d9dd759f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.098294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba72497-1ca0-4bdf-bcb8-8797d9dd759f admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.099005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba72497-1ca0-4bdf-bcb8-8797d9dd759f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:55.101007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba72497-1ca0-4bdf-bcb8-8797d9dd759f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:55.111664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:55.135674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 987/1977] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:54 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:04:55.136923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba72497-1ca0-4bdf-bcb8-8797d9dd759f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:55.139896 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-eba72497-1ca0-4bdf-bcb8-8797d9dd759f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:55.160777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba72497-1ca0-4bdf-bcb8-8797d9dd759f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:55.163295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e1bb9-8fdc-482f-b6ea-76eb2083ee44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:55.164002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e1bb9-8fdc-482f-b6ea-76eb2083ee44 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.164288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e1bb9-8fdc-482f-b6ea-76eb2083ee44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.164577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e1bb9-8fdc-482f-b6ea-76eb2083ee44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.166945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab339053-347f-4d21-b3c3-842277233f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:55.168147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab339053-347f-4d21-b3c3-842277233f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.168571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab339053-347f-4d21-b3c3-842277233f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.169000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab339053-347f-4d21-b3c3-842277233f10 admin admin] PATH_INFO: `/v3/projects/b433490dc30e49ef8dc2b7d13416f5e0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.170585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab339053-347f-4d21-b3c3-842277233f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b433490dc30e49ef8dc2b7d13416f5e0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:55.172820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab339053-347f-4d21-b3c3-842277233f10 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:55.176145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e1bb9-8fdc-482f-b6ea-76eb2083ee44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:55.177709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e1bb9-8fdc-482f-b6ea-76eb2083ee44 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:55.188666 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 991/1978] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:55.189638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e1bb9-8fdc-482f-b6ea-76eb2083ee44 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:55.200250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e1bb9-8fdc-482f-b6ea-76eb2083ee44 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:55.205600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e3f9c-0a5d-4893-b7aa-ac1ccace7a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:55.206582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e3f9c-0a5d-4893-b7aa-ac1ccace7a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.206849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e3f9c-0a5d-4893-b7aa-ac1ccace7a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.207156 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e3f9c-0a5d-4893-b7aa-ac1ccace7a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.208024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e3f9c-0a5d-4893-b7aa-ac1ccace7a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:55.209598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e3f9c-0a5d-4893-b7aa-ac1ccace7a87 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:55.221863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 992/1979] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:55.228148 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e3f9c-0a5d-4893-b7aa-ac1ccace7a87 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:55.240060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e3f9c-0a5d-4893-b7aa-ac1ccace7a87 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:55.264280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab339053-347f-4d21-b3c3-842277233f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b433490dc30e49ef8dc2b7d13416f5e0', 'request_id': 'req-ab339053-347f-4d21-b3c3-842277233f10'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:55.277746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab339053-347f-4d21-b3c3-842277233f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b433490dc30e49ef8dc2b7d13416f5e0', 'request_id': 'req-ab339053-347f-4d21-b3c3-842277233f10'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:55.291975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab339053-347f-4d21-b3c3-842277233f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b433490dc30e49ef8dc2b7d13416f5e0', 'request_id': 'req-ab339053-347f-4d21-b3c3-842277233f10'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:55.316232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab339053-347f-4d21-b3c3-842277233f10 admin admin] The token cache is being invalidate because project b433490dc30e49ef8dc2b7d13416f5e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:55.316232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab339053-347f-4d21-b3c3-842277233f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:55.317187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab339053-347f-4d21-b3c3-842277233f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:55.319832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 988/1980] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:55 2022] DELETE /identity/v3/projects/b433490dc30e49ef8dc2b7d13416f5e0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:55.337123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b831ab3-43b1-48a2-9adc-2def7f199b0a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.337412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b831ab3-43b1-48a2-9adc-2def7f199b0a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.337729 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b831ab3-43b1-48a2-9adc-2def7f199b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.352376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce7365-7858-4797-a671-f6e6381f8cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:55.353076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce7365-7858-4797-a671-f6e6381f8cba admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.353360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce7365-7858-4797-a671-f6e6381f8cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.353654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce7365-7858-4797-a671-f6e6381f8cba admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.354409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce7365-7858-4797-a671-f6e6381f8cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:55.356065 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce7365-7858-4797-a671-f6e6381f8cba admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:55.358689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8b831ab3-43b1-48a2-9adc-2def7f199b0a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:55.359968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8b831ab3-43b1-48a2-9adc-2def7f199b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:55.369494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 989/1981] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:55.379194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be7d6f-961c-492d-a603-8c838ec3decc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.379640 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 993/1982] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:55.382674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be7d6f-961c-492d-a603-8c838ec3decc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.383550 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be7d6f-961c-492d-a603-8c838ec3decc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.385477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce7365-7858-4797-a671-f6e6381f8cba admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:55.388227 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a6ce7365-7858-4797-a671-f6e6381f8cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:55.405119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-04be7d6f-961c-492d-a603-8c838ec3decc None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:55.405567 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce7365-7858-4797-a671-f6e6381f8cba admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:55.406709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-04be7d6f-961c-492d-a603-8c838ec3decc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:55.416331 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30981d5-4d77-4d72-a806-94061bca95b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:55.417480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30981d5-4d77-4d72-a806-94061bca95b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.417964 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30981d5-4d77-4d72-a806-94061bca95b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.418194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30981d5-4d77-4d72-a806-94061bca95b1 admin admin] PATH_INFO: `/v3/projects/f2416194316d4088b02f35a6673dd007/users/e67aa374e7874bd6b484c6180c1a5deb/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.419130 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 990/1983] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:55.426902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be7d6f-961c-492d-a603-8c838ec3decc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:55.438580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:55.442979 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be7d6f-961c-492d-a603-8c838ec3decc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:55.455344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd21d2-839b-4168-bfcc-3b9c97224bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:55.456535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd21d2-839b-4168-bfcc-3b9c97224bee admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.457003 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd21d2-839b-4168-bfcc-3b9c97224bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.457428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd21d2-839b-4168-bfcc-3b9c97224bee admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.458937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd21d2-839b-4168-bfcc-3b9c97224bee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:04:55.459515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd21d2-839b-4168-bfcc-3b9c97224bee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:55.462704 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd21d2-839b-4168-bfcc-3b9c97224bee admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:55.474495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30981d5-4d77-4d72-a806-94061bca95b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2416194316d4088b02f35a6673dd007, user_id=e67aa374e7874bd6b484c6180c1a5deb, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:55.481950 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30981d5-4d77-4d72-a806-94061bca95b1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:55.484733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 991/1984] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:04:55.490057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd21d2-839b-4168-bfcc-3b9c97224bee admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:55.504197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 994/1985] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:55 2022] PUT /identity/v3/projects/f2416194316d4088b02f35a6673dd007/users/e67aa374e7874bd6b484c6180c1a5deb/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:55.506339 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd21d2-839b-4168-bfcc-3b9c97224bee admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:55.519180 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d974d3-e6f1-4f17-8509-9b3993d5ff59 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.519531 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d974d3-e6f1-4f17-8509-9b3993d5ff59 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.519742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d974d3-e6f1-4f17-8509-9b3993d5ff59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.556363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c0d974d3-e6f1-4f17-8509-9b3993d5ff59 None None] MFA Rules not processed for user `e67aa374e7874bd6b484c6180c1a5deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:55.558256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c0d974d3-e6f1-4f17-8509-9b3993d5ff59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:55.620941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563bf4e-81b6-4215-bab6-b2ffc87099e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:55.622028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563bf4e-81b6-4215-bab6-b2ffc87099e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.622561 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563bf4e-81b6-4215-bab6-b2ffc87099e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.623112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563bf4e-81b6-4215-bab6-b2ffc87099e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.624054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563bf4e-81b6-4215-bab6-b2ffc87099e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:55.626415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563bf4e-81b6-4215-bab6-b2ffc87099e7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:55.668732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 992/1986] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:55 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:04:55.668732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563bf4e-81b6-4215-bab6-b2ffc87099e7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:55.670632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 995/1987] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:55.671828 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d974d3-e6f1-4f17-8509-9b3993d5ff59 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:55.683203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b563bf4e-81b6-4215-bab6-b2ffc87099e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:55.693921 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d974d3-e6f1-4f17-8509-9b3993d5ff59 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:55.705028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:55.778116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039aefc3-2141-4b6a-8be3-03b5d2cbf6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:55.778725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039aefc3-2141-4b6a-8be3-03b5d2cbf6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.779005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039aefc3-2141-4b6a-8be3-03b5d2cbf6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.779321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039aefc3-2141-4b6a-8be3-03b5d2cbf6c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.779997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039aefc3-2141-4b6a-8be3-03b5d2cbf6c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:55.782160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039aefc3-2141-4b6a-8be3-03b5d2cbf6c2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:55.814366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 996/1988] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:55 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:55.815914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039aefc3-2141-4b6a-8be3-03b5d2cbf6c2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:55.820401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-039aefc3-2141-4b6a-8be3-03b5d2cbf6c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:55.829061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563bf4e-81b6-4215-bab6-b2ffc87099e7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:55.838830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:55.840402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.840923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.841684 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] PATH_INFO: `/v3/projects/bda6d17f355e47c9a675ce3bf595bf15` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.844197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda6d17f355e47c9a675ce3bf595bf15)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:55.848153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:55.905652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039aefc3-2141-4b6a-8be3-03b5d2cbf6c2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:55.910491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d452a-2ec5-488e-97e8-2daf2e9b0302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:55.911176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d452a-2ec5-488e-97e8-2daf2e9b0302 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.911462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d452a-2ec5-488e-97e8-2daf2e9b0302 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:55.911747 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d452a-2ec5-488e-97e8-2daf2e9b0302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:55.938991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda6d17f355e47c9a675ce3bf595bf15', 'request_id': 'req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:55.946644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda6d17f355e47c9a675ce3bf595bf15', 'request_id': 'req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:55.957325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda6d17f355e47c9a675ce3bf595bf15', 'request_id': 'req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:55.971712 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] The token cache is being invalidate because project bda6d17f355e47c9a675ce3bf595bf15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:04:55.972193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:55.973241 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:55.975621 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 993/1989] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:04:55 2022] DELETE /identity/v3/projects/bda6d17f355e47c9a675ce3bf595bf15 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:55.976827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:55.979298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:55.980805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d452a-2ec5-488e-97e8-2daf2e9b0302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:55.982373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d452a-2ec5-488e-97e8-2daf2e9b0302 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:55.991987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:55.996801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 997/1990] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:55.999317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f460b-7b9b-4e67-bbb4-21a68797626c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:55.999820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f460b-7b9b-4e67-bbb4-21a68797626c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.000295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f460b-7b9b-4e67-bbb4-21a68797626c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.037110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f66f460b-7b9b-4e67-bbb4-21a68797626c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:56.039543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f66f460b-7b9b-4e67-bbb4-21a68797626c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:56.069731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd9fa2-aaa9-449e-b58b-53b9a203618d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.074968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4bcc7-fca6-4044-988b-196c2c15ff6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.075678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4bcc7-fca6-4044-988b-196c2c15ff6a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.075961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4bcc7-fca6-4044-988b-196c2c15ff6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.076237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4bcc7-fca6-4044-988b-196c2c15ff6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.077453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4bcc7-fca6-4044-988b-196c2c15ff6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.078850 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4bcc7-fca6-4044-988b-196c2c15ff6a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.094686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 994/1991] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:56.096928 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4bcc7-fca6-4044-988b-196c2c15ff6a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:56.120427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4bcc7-fca6-4044-988b-196c2c15ff6a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.126346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67a3c5-a786-4a19-bee6-0a94604adec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.127471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67a3c5-a786-4a19-bee6-0a94604adec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.128001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67a3c5-a786-4a19-bee6-0a94604adec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.128849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67a3c5-a786-4a19-bee6-0a94604adec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.129745 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67a3c5-a786-4a19-bee6-0a94604adec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.131606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67a3c5-a786-4a19-bee6-0a94604adec4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.143710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:56.150132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 995/1992] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:04:56.151524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67a3c5-a786-4a19-bee6-0a94604adec4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:56.173743 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67a3c5-a786-4a19-bee6-0a94604adec4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.174060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 998/1993] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:56.182650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e76071-416a-4a0f-b511-8ef22d8d83f6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.183025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e76071-416a-4a0f-b511-8ef22d8d83f6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.183228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e76071-416a-4a0f-b511-8ef22d8d83f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.188642 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b728d03-442c-494d-b70e-a9173ba2696d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.190322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b728d03-442c-494d-b70e-a9173ba2696d admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.191078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b728d03-442c-494d-b70e-a9173ba2696d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.191667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b728d03-442c-494d-b70e-a9173ba2696d admin admin] PATH_INFO: `/v3/projects/be961116123546c9a6d504cb1e6d7747/users/63fcd3ec034443c7a5b5423140d32688/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.216655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c0e76071-416a-4a0f-b511-8ef22d8d83f6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:56.218867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c0e76071-416a-4a0f-b511-8ef22d8d83f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:56.221364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b728d03-442c-494d-b70e-a9173ba2696d admin admin] RBAC: Authorizing `identity:create_grant(project_id=be961116123546c9a6d504cb1e6d7747, user_id=63fcd3ec034443c7a5b5423140d32688, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.225887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b728d03-442c-494d-b70e-a9173ba2696d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.233953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 999/1994] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:56.241179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e76071-416a-4a0f-b511-8ef22d8d83f6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:56.248899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 996/1995] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:56 2022] PUT /identity/v3/projects/be961116123546c9a6d504cb1e6d7747/users/63fcd3ec034443c7a5b5423140d32688/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:56.260298 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e76071-416a-4a0f-b511-8ef22d8d83f6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.270965 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bde2e-e05c-41f6-9718-b31c9b6167f1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.271450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bde2e-e05c-41f6-9718-b31c9b6167f1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.271935 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bde2e-e05c-41f6-9718-b31c9b6167f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.335120 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-702bde2e-e05c-41f6-9718-b31c9b6167f1 None None] MFA Rules not processed for user `63fcd3ec034443c7a5b5423140d32688`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:56.337414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-702bde2e-e05c-41f6-9718-b31c9b6167f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:56.354588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2aee09-6f71-456b-9eb1-515d9b83631e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.356448 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2aee09-6f71-456b-9eb1-515d9b83631e admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.356956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2aee09-6f71-456b-9eb1-515d9b83631e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.357452 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2aee09-6f71-456b-9eb1-515d9b83631e admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.359179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2aee09-6f71-456b-9eb1-515d9b83631e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:04:56.359780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2aee09-6f71-456b-9eb1-515d9b83631e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.364102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2aee09-6f71-456b-9eb1-515d9b83631e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.389801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1000/1996] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:04:56.397413 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2aee09-6f71-456b-9eb1-515d9b83631e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:56.431026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2aee09-6f71-456b-9eb1-515d9b83631e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.434667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 997/1997] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:56.439601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702bde2e-e05c-41f6-9718-b31c9b6167f1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:56.450718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cda336-d01d-4757-9c65-de7eaca08cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.452569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cda336-d01d-4757-9c65-de7eaca08cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.453165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cda336-d01d-4757-9c65-de7eaca08cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.453327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702bde2e-e05c-41f6-9718-b31c9b6167f1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.454039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cda336-d01d-4757-9c65-de7eaca08cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.456160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cda336-d01d-4757-9c65-de7eaca08cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.458873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50420d01-6823-4868-8275-296a4fb97a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.459576 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50420d01-6823-4868-8275-296a4fb97a12 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.459863 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50420d01-6823-4868-8275-296a4fb97a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.460264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50420d01-6823-4868-8275-296a4fb97a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.461075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50420d01-6823-4868-8275-296a4fb97a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.461508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cda336-d01d-4757-9c65-de7eaca08cc6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.463478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50420d01-6823-4868-8275-296a4fb97a12 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.499541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 998/1998] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:56 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:04:56.505441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50420d01-6823-4868-8275-296a4fb97a12 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:56.533338 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1001/1999] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:56 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:04:56.538674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cda336-d01d-4757-9c65-de7eaca08cc6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:56.583428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:56.635123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50420d01-6823-4868-8275-296a4fb97a12 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.686523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cda336-d01d-4757-9c65-de7eaca08cc6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.695287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ec896-bd01-491f-bc77-d0e9191d6ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.696651 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ec896-bd01-491f-bc77-d0e9191d6ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.696945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ec896-bd01-491f-bc77-d0e9191d6ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.697166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ec896-bd01-491f-bc77-d0e9191d6ca1 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.698492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ec896-bd01-491f-bc77-d0e9191d6ca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.700576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ec896-bd01-491f-bc77-d0e9191d6ca1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.741682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1002/2000] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:56 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:56.743108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ec896-bd01-491f-bc77-d0e9191d6ca1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:56.758686 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ec896-bd01-491f-bc77-d0e9191d6ca1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.764150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c063a9-cd9a-40cf-a075-cd8808258234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.764850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c063a9-cd9a-40cf-a075-cd8808258234 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.765203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c063a9-cd9a-40cf-a075-cd8808258234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.765456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c063a9-cd9a-40cf-a075-cd8808258234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.767206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c063a9-cd9a-40cf-a075-cd8808258234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.768046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c063a9-cd9a-40cf-a075-cd8808258234 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.769972 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365050a-dfa0-456b-9cb5-aa54eca596a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.770761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365050a-dfa0-456b-9cb5-aa54eca596a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.771041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365050a-dfa0-456b-9cb5-aa54eca596a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.771322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365050a-dfa0-456b-9cb5-aa54eca596a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.772200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365050a-dfa0-456b-9cb5-aa54eca596a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.774307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365050a-dfa0-456b-9cb5-aa54eca596a1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.786567 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1003/2001] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:04:56.788165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c063a9-cd9a-40cf-a075-cd8808258234 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:56.801117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c063a9-cd9a-40cf-a075-cd8808258234 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.805999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 999/2002] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:56 2022] POST /identity/v3/users => generated 298 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:04:56.807409 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365050a-dfa0-456b-9cb5-aa54eca596a1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:56.812309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d359c67-50cf-4e78-a5b4-608c9b2fad42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.813231 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d359c67-50cf-4e78-a5b4-608c9b2fad42 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.813724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d359c67-50cf-4e78-a5b4-608c9b2fad42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.814018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d359c67-50cf-4e78-a5b4-608c9b2fad42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.816092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d359c67-50cf-4e78-a5b4-608c9b2fad42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.816581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d359c67-50cf-4e78-a5b4-608c9b2fad42 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.821874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365050a-dfa0-456b-9cb5-aa54eca596a1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.828875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb353888-f640-48e5-8151-6fb30f05f501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.829831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb353888-f640-48e5-8151-6fb30f05f501 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.830247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb353888-f640-48e5-8151-6fb30f05f501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.830452 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb353888-f640-48e5-8151-6fb30f05f501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.831568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb353888-f640-48e5-8151-6fb30f05f501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.834370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb353888-f640-48e5-8151-6fb30f05f501 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.837486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1004/2003] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:56.841803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d359c67-50cf-4e78-a5b4-608c9b2fad42 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:56.853747 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1000/2004] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:04:56.854696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d359c67-50cf-4e78-a5b4-608c9b2fad42 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.860128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504193e-88b6-46af-b534-e6f65b7f320d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.860988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb353888-f640-48e5-8151-6fb30f05f501 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:56.861197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504193e-88b6-46af-b534-e6f65b7f320d admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.861360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504193e-88b6-46af-b534-e6f65b7f320d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.861682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504193e-88b6-46af-b534-e6f65b7f320d admin admin] PATH_INFO: `/v3/projects/01e901d5c0f24344b41fceee0c9d1f7d/users/f9602032a09f4a17b1844762edffb024/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.883648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb353888-f640-48e5-8151-6fb30f05f501 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:56.887931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504193e-88b6-46af-b534-e6f65b7f320d admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e901d5c0f24344b41fceee0c9d1f7d, user_id=f9602032a09f4a17b1844762edffb024, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.891944 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504193e-88b6-46af-b534-e6f65b7f320d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.893884 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114506fb-ceca-4d4a-985d-bf62c7368014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:56.895458 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114506fb-ceca-4d4a-985d-bf62c7368014 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.896004 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114506fb-ceca-4d4a-985d-bf62c7368014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.896458 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114506fb-ceca-4d4a-985d-bf62c7368014 admin admin] PATH_INFO: `/v3/projects/0e4a8831c7b74b84a490b9e53575d655/users/02ef2b682265447f92c71e14ec1692af/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.916868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1005/2005] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:56 2022] PUT /identity/v3/projects/01e901d5c0f24344b41fceee0c9d1f7d/users/f9602032a09f4a17b1844762edffb024/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:56.931656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6113acc-b613-4627-926c-6d933fea63ca None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:56.932417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6113acc-b613-4627-926c-6d933fea63ca None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:56.933078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6113acc-b613-4627-926c-6d933fea63ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:56.949547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114506fb-ceca-4d4a-985d-bf62c7368014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e4a8831c7b74b84a490b9e53575d655, user_id=02ef2b682265447f92c71e14ec1692af, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:56.955197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114506fb-ceca-4d4a-985d-bf62c7368014 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:56.977830 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:56.982742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1001/2006] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:56 2022] PUT /identity/v3/projects/0e4a8831c7b74b84a490b9e53575d655/users/02ef2b682265447f92c71e14ec1692af/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:56.990320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114506fb-ceca-4d4a-985d-bf62c7368014 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:57.019303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a6113acc-b613-4627-926c-6d933fea63ca None None] MFA Rules not processed for user `f9602032a09f4a17b1844762edffb024`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:57.021655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a6113acc-b613-4627-926c-6d933fea63ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:57.143518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114506fb-ceca-4d4a-985d-bf62c7368014 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:57.154734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d918ac-a0fa-4f33-a9de-a6cba25f8cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:57.155883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d918ac-a0fa-4f33-a9de-a6cba25f8cdd admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:57.156409 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d918ac-a0fa-4f33-a9de-a6cba25f8cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:57.156936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d918ac-a0fa-4f33-a9de-a6cba25f8cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:57.158650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d918ac-a0fa-4f33-a9de-a6cba25f8cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:57.161946 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d918ac-a0fa-4f33-a9de-a6cba25f8cdd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:57.174691 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1006/2007] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:57.184215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6113acc-b613-4627-926c-6d933fea63ca None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:57.186868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1002/2008] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:04:57.191999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d918ac-a0fa-4f33-a9de-a6cba25f8cdd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:57.204208 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6113acc-b613-4627-926c-6d933fea63ca None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:57.212517 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d918ac-a0fa-4f33-a9de-a6cba25f8cdd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:57.213188 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995ca9f-ad00-4d98-830c-86dd4576e28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:57.214404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995ca9f-ad00-4d98-830c-86dd4576e28d admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:57.215052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995ca9f-ad00-4d98-830c-86dd4576e28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:57.215501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995ca9f-ad00-4d98-830c-86dd4576e28d admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:57.217132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995ca9f-ad00-4d98-830c-86dd4576e28d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:57.221559 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995ca9f-ad00-4d98-830c-86dd4576e28d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:57.221796 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd244d-7c49-4bd4-b8ac-68d893e37a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:57.223560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd244d-7c49-4bd4-b8ac-68d893e37a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:57.224059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd244d-7c49-4bd4-b8ac-68d893e37a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:57.224566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd244d-7c49-4bd4-b8ac-68d893e37a5f admin admin] PATH_INFO: `/v3/projects/0e4a8831c7b74b84a490b9e53575d655/users/02ef2b682265447f92c71e14ec1692af/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:57.229119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd244d-7c49-4bd4-b8ac-68d893e37a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e4a8831c7b74b84a490b9e53575d655, user_id=02ef2b682265447f92c71e14ec1692af, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:57.234772 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd244d-7c49-4bd4-b8ac-68d893e37a5f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:57.256079 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1003/2009] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:57 2022] PUT /identity/v3/projects/0e4a8831c7b74b84a490b9e53575d655/users/02ef2b682265447f92c71e14ec1692af/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:04:57.258431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1007/2010] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:04:57 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:04:57.263966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd244d-7c49-4bd4-b8ac-68d893e37a5f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:57.276112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28c146-b8a9-4292-ac63-f61c23354c47 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:57.276446 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28c146-b8a9-4292-ac63-f61c23354c47 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:57.276823 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28c146-b8a9-4292-ac63-f61c23354c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:57.298844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:57.325902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5e28c146-b8a9-4292-ac63-f61c23354c47 None None] MFA Rules not processed for user `02ef2b682265447f92c71e14ec1692af`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:57.328308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5e28c146-b8a9-4292-ac63-f61c23354c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:57.375359 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd244d-7c49-4bd4-b8ac-68d893e37a5f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:57.437872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1008/2011] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:57.450867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28c146-b8a9-4292-ac63-f61c23354c47 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:57.452398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5e28c146-b8a9-4292-ac63-f61c23354c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:57.461324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c39240-f656-4973-9015-89cd32e7f151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:57.462273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c39240-f656-4973-9015-89cd32e7f151 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:57.462386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c39240-f656-4973-9015-89cd32e7f151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:57.462772 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c39240-f656-4973-9015-89cd32e7f151 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:57.463522 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c39240-f656-4973-9015-89cd32e7f151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:57.465554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c39240-f656-4973-9015-89cd32e7f151 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:57.498869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1004/2012] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:04:57 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:04:57.501871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c39240-f656-4973-9015-89cd32e7f151 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:57.522081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c39240-f656-4973-9015-89cd32e7f151 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:57.529996 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79741458-9f37-46dd-bb21-d1e5edafb95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:57.531417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79741458-9f37-46dd-bb21-d1e5edafb95a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:57.531894 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79741458-9f37-46dd-bb21-d1e5edafb95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:57.532305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79741458-9f37-46dd-bb21-d1e5edafb95a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:57.535048 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79741458-9f37-46dd-bb21-d1e5edafb95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:57.539505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79741458-9f37-46dd-bb21-d1e5edafb95a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:57.541941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28c146-b8a9-4292-ac63-f61c23354c47 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:57.550719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc456e1-eab8-4c91-b35e-5dbf18b79231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:57.551918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc456e1-eab8-4c91-b35e-5dbf18b79231 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:57.552379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc456e1-eab8-4c91-b35e-5dbf18b79231 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:57.552866 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc456e1-eab8-4c91-b35e-5dbf18b79231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:57.572761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc456e1-eab8-4c91-b35e-5dbf18b79231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:57.575714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc456e1-eab8-4c91-b35e-5dbf18b79231 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:57.577849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1005/2013] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:57.580222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79741458-9f37-46dd-bb21-d1e5edafb95a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:57.597111 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1009/2014] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:04:57.604288 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79741458-9f37-46dd-bb21-d1e5edafb95a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:57.609985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086e20f-8e2e-43a4-876c-081842cafbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:57.611006 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086e20f-8e2e-43a4-876c-081842cafbfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:57.611363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086e20f-8e2e-43a4-876c-081842cafbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:57.611756 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086e20f-8e2e-43a4-876c-081842cafbfe admin admin] PATH_INFO: `/v3/projects/35e2d940ed954d109e7eec97f5bd4dbc/users/83c2b077048d4ec5a818c7b2c062422c/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:57.639333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086e20f-8e2e-43a4-876c-081842cafbfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e2d940ed954d109e7eec97f5bd4dbc, user_id=83c2b077048d4ec5a818c7b2c062422c, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:57.641372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086e20f-8e2e-43a4-876c-081842cafbfe admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:57.677137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1006/2015] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:57 2022] PUT /identity/v3/projects/35e2d940ed954d109e7eec97f5bd4dbc/users/83c2b077048d4ec5a818c7b2c062422c/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:57.678824 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc456e1-eab8-4c91-b35e-5dbf18b79231 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:57.684922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:57.699849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc456e1-eab8-4c91-b35e-5dbf18b79231 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:57.773449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086e20f-8e2e-43a4-876c-081842cafbfe admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:57.857265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b3305-3ac8-40ea-9d62-abb4d40b6e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:57.858018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b3305-3ac8-40ea-9d62-abb4d40b6e44 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:57.858306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b3305-3ac8-40ea-9d62-abb4d40b6e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:57.858585 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b3305-3ac8-40ea-9d62-abb4d40b6e44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:57.859351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b3305-3ac8-40ea-9d62-abb4d40b6e44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:57.863933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b3305-3ac8-40ea-9d62-abb4d40b6e44 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:57.884229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1010/2016] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:04:57.899417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b3305-3ac8-40ea-9d62-abb4d40b6e44 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:57.919092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b3305-3ac8-40ea-9d62-abb4d40b6e44 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:57.924753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3492b-371a-40f4-b8ad-a9390ccb8501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:57.925775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3492b-371a-40f4-b8ad-a9390ccb8501 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:57.926235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3492b-371a-40f4-b8ad-a9390ccb8501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:57.926417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3492b-371a-40f4-b8ad-a9390ccb8501 admin admin] PATH_INFO: `/v3/projects/35e2d940ed954d109e7eec97f5bd4dbc/users/83c2b077048d4ec5a818c7b2c062422c/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:57.932590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3492b-371a-40f4-b8ad-a9390ccb8501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e2d940ed954d109e7eec97f5bd4dbc, user_id=83c2b077048d4ec5a818c7b2c062422c, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:57.939012 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3492b-371a-40f4-b8ad-a9390ccb8501 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:57.955197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1011/2017] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:04:57 2022] PUT /identity/v3/projects/35e2d940ed954d109e7eec97f5bd4dbc/users/83c2b077048d4ec5a818c7b2c062422c/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:57.975416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af3f4b-dca9-4f76-88df-f9420e49d25e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:57.975929 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af3f4b-dca9-4f76-88df-f9420e49d25e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:57.976431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af3f4b-dca9-4f76-88df-f9420e49d25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:57.980213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086e20f-8e2e-43a4-876c-081842cafbfe admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:58.033237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:58.051442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-90af3f4b-dca9-4f76-88df-f9420e49d25e None None] MFA Rules not processed for user `83c2b077048d4ec5a818c7b2c062422c`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:58.054309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-90af3f4b-dca9-4f76-88df-f9420e49d25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:58.076247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd5215-63fe-4804-be48-f99ab38f74d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:58.077424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd5215-63fe-4804-be48-f99ab38f74d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:58.078152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd5215-63fe-4804-be48-f99ab38f74d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:58.078796 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd5215-63fe-4804-be48-f99ab38f74d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:58.155517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1012/2018] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:04:58.163555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd5215-63fe-4804-be48-f99ab38f74d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:58.165046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd5215-63fe-4804-be48-f99ab38f74d5 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:58.175702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1007/2019] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:04:58.283549 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af3f4b-dca9-4f76-88df-f9420e49d25e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:58.299441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af3f4b-dca9-4f76-88df-f9420e49d25e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:58.304286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e69f3-a9e4-45bd-b4bb-f39b0bc0c1a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:58.304963 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e69f3-a9e4-45bd-b4bb-f39b0bc0c1a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:58.305414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e69f3-a9e4-45bd-b4bb-f39b0bc0c1a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:58.305577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e69f3-a9e4-45bd-b4bb-f39b0bc0c1a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:58.396080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e69f3-a9e4-45bd-b4bb-f39b0bc0c1a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:58.397585 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e69f3-a9e4-45bd-b4bb-f39b0bc0c1a4 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:58.409936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1013/2020] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:04:59.201964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd5215-63fe-4804-be48-f99ab38f74d5 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:59.310281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd5215-63fe-4804-be48-f99ab38f74d5 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:59.318753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:59.319773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:59.320117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:59.320450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] PATH_INFO: `/v3/users/63fcd3ec034443c7a5b5423140d32688` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:59.322282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=63fcd3ec034443c7a5b5423140d32688)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:59.325148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:59.392563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63fcd3ec034443c7a5b5423140d32688'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:59.412575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63fcd3ec034443c7a5b5423140d32688'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:59.424623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63fcd3ec034443c7a5b5423140d32688'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:59.425557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63fcd3ec034443c7a5b5423140d32688'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:59.444628 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63fcd3ec034443c7a5b5423140d32688'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:04:59.453103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1008/2021] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:04:59 2022] DELETE /identity/v3/users/63fcd3ec034443c7a5b5423140d32688 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:04:59.458421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9bebd-d6dd-4d55-90c4-b8a27e7ecf0f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:59.459227 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9bebd-d6dd-4d55-90c4-b8a27e7ecf0f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:59.459375 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9bebd-d6dd-4d55-90c4-b8a27e7ecf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:59.480042 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a1d9bebd-d6dd-4d55-90c4-b8a27e7ecf0f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:04:59.481296 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a1d9bebd-d6dd-4d55-90c4-b8a27e7ecf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:59.486435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:59.561509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1014/2022] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:04:59.574343 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:04:59.576801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:04:59.594542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:04:59.677289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386a549-dc33-4331-af85-9008bbc91d2a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:04:59.682513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2f987-1c6f-4ccd-b899-5e01a8bf34d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:04:59.683197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2f987-1c6f-4ccd-b899-5e01a8bf34d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:04:59.683605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2f987-1c6f-4ccd-b899-5e01a8bf34d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:04:59.683747 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2f987-1c6f-4ccd-b899-5e01a8bf34d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:04:59.701278 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2f987-1c6f-4ccd-b899-5e01a8bf34d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:04:59.703885 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2f987-1c6f-4ccd-b899-5e01a8bf34d4 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:04:59.724971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1009/2023] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:04:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:00.189479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9bebd-d6dd-4d55-90c4-b8a27e7ecf0f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:00.191972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a1d9bebd-d6dd-4d55-90c4-b8a27e7ecf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:00.217584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9bebd-d6dd-4d55-90c4-b8a27e7ecf0f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:00.224607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:00.225947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:00.226971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:00.227793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] PATH_INFO: `/v3/projects/be961116123546c9a6d504cb1e6d7747` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:00.230570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] RBAC: Authorizing `identity:delete_project(project_id=be961116123546c9a6d504cb1e6d7747)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:00.233248 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:00.321016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be961116123546c9a6d504cb1e6d7747', 'request_id': 'req-56c2977e-b770-4685-b131-68aa50b6951a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:00.346509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be961116123546c9a6d504cb1e6d7747', 'request_id': 'req-56c2977e-b770-4685-b131-68aa50b6951a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:00.357290 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be961116123546c9a6d504cb1e6d7747', 'request_id': 'req-56c2977e-b770-4685-b131-68aa50b6951a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:00.383155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] The token cache is being invalidate because project be961116123546c9a6d504cb1e6d7747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:00.383550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:00.384554 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:00.387161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1015/2024] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:00 2022] DELETE /identity/v3/projects/be961116123546c9a6d504cb1e6d7747 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:00.393078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2f987-1c6f-4ccd-b899-5e01a8bf34d4 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:00.410827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2f987-1c6f-4ccd-b899-5e01a8bf34d4 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:00.486384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dea6d-1e46-4956-8852-fc604db45148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:00.487481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dea6d-1e46-4956-8852-fc604db45148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:00.487765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dea6d-1e46-4956-8852-fc604db45148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:00.488105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dea6d-1e46-4956-8852-fc604db45148 admin admin] PATH_INFO: `/v3/users/02ef2b682265447f92c71e14ec1692af` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:00.489848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dea6d-1e46-4956-8852-fc604db45148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02ef2b682265447f92c71e14ec1692af)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:00.492135 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dea6d-1e46-4956-8852-fc604db45148 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:00.565827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-109dea6d-1e46-4956-8852-fc604db45148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02ef2b682265447f92c71e14ec1692af'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:00.578907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-109dea6d-1e46-4956-8852-fc604db45148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02ef2b682265447f92c71e14ec1692af'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:00.590061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-109dea6d-1e46-4956-8852-fc604db45148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02ef2b682265447f92c71e14ec1692af'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:00.592660 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-109dea6d-1e46-4956-8852-fc604db45148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02ef2b682265447f92c71e14ec1692af'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:00.614330 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-109dea6d-1e46-4956-8852-fc604db45148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02ef2b682265447f92c71e14ec1692af'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:00.625118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1010/2025] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:00 2022] DELETE /identity/v3/users/02ef2b682265447f92c71e14ec1692af => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:01.072182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:01.078610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:01.093459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2977e-b770-4685-b131-68aa50b6951a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:01.185247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:01.187016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:01.187470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:01.187919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] PATH_INFO: `/v3/projects/0e4a8831c7b74b84a490b9e53575d655` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:01.190126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e4a8831c7b74b84a490b9e53575d655)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:01.193616 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:01.264119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:01.284675 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e4a8831c7b74b84a490b9e53575d655', 'request_id': 'req-e97bdbf5-e189-4a55-854b-236a8799f820'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:01.296630 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e4a8831c7b74b84a490b9e53575d655', 'request_id': 'req-e97bdbf5-e189-4a55-854b-236a8799f820'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:01.310327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e4a8831c7b74b84a490b9e53575d655', 'request_id': 'req-e97bdbf5-e189-4a55-854b-236a8799f820'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:01.340360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] The token cache is being invalidate because project 0e4a8831c7b74b84a490b9e53575d655 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:01.341228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:01.342563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:01.349865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1016/2026] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:01 2022] DELETE /identity/v3/projects/0e4a8831c7b74b84a490b9e53575d655 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:01.353468 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eff48-5346-40bf-ace8-a3f12af0c31e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:01.353796 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eff48-5346-40bf-ace8-a3f12af0c31e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:01.354092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eff48-5346-40bf-ace8-a3f12af0c31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:01.375735 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-620eff48-5346-40bf-ace8-a3f12af0c31e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:01.378104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-620eff48-5346-40bf-ace8-a3f12af0c31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:01.423109 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:01.469724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:01.472943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:01.500790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1011/2027] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:05:01.514402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a1b9b-3cb9-4bd1-8c99-ccd0aead8fb5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:01.515199 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a1b9b-3cb9-4bd1-8c99-ccd0aead8fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:01.515659 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a1b9b-3cb9-4bd1-8c99-ccd0aead8fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:01.541222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-393a1b9b-3cb9-4bd1-8c99-ccd0aead8fb5 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:01.542929 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-393a1b9b-3cb9-4bd1-8c99-ccd0aead8fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:01.557849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1012/2028] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:01.559192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a1b9b-3cb9-4bd1-8c99-ccd0aead8fb5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:01.571519 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a1b9b-3cb9-4bd1-8c99-ccd0aead8fb5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:01.579081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e98a6-7509-4411-8ece-26b7e48b9693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:01.580208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e98a6-7509-4411-8ece-26b7e48b9693 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:01.580784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e98a6-7509-4411-8ece-26b7e48b9693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:01.581261 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e98a6-7509-4411-8ece-26b7e48b9693 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:01.582322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e98a6-7509-4411-8ece-26b7e48b9693 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:05:01.582821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e98a6-7509-4411-8ece-26b7e48b9693 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:01.584824 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e98a6-7509-4411-8ece-26b7e48b9693 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:01.595169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bdbf5-e189-4a55-854b-236a8799f820 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:01.602891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39419399-7083-4cf7-80c8-278695b0acea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:01.607604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39419399-7083-4cf7-80c8-278695b0acea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:01.607604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39419399-7083-4cf7-80c8-278695b0acea admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:01.607604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39419399-7083-4cf7-80c8-278695b0acea admin admin] PATH_INFO: `/v3/users/f9602032a09f4a17b1844762edffb024` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:01.607966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1013/2029] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:05:01.608072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39419399-7083-4cf7-80c8-278695b0acea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9602032a09f4a17b1844762edffb024)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:01.609992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e98a6-7509-4411-8ece-26b7e48b9693 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:01.610354 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39419399-7083-4cf7-80c8-278695b0acea admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:01.629274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e98a6-7509-4411-8ece-26b7e48b9693 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:01.636560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5329a2e-16ed-4e85-9942-38deabad07de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:01.637583 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5329a2e-16ed-4e85-9942-38deabad07de admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:01.638342 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5329a2e-16ed-4e85-9942-38deabad07de admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:01.638781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5329a2e-16ed-4e85-9942-38deabad07de admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:01.640329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5329a2e-16ed-4e85-9942-38deabad07de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:01.643977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5329a2e-16ed-4e85-9942-38deabad07de admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:01.699935 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1014/2030] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:05:01 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:05:01.702309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5329a2e-16ed-4e85-9942-38deabad07de admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:01.705165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-39419399-7083-4cf7-80c8-278695b0acea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9602032a09f4a17b1844762edffb024'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:01.706493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-39419399-7083-4cf7-80c8-278695b0acea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9602032a09f4a17b1844762edffb024'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:01.707827 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-39419399-7083-4cf7-80c8-278695b0acea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9602032a09f4a17b1844762edffb024'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:01.720803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-39419399-7083-4cf7-80c8-278695b0acea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9602032a09f4a17b1844762edffb024'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:01.757152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-39419399-7083-4cf7-80c8-278695b0acea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9602032a09f4a17b1844762edffb024'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:01.786788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1017/2031] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:01 2022] DELETE /identity/v3/users/f9602032a09f4a17b1844762edffb024 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:05:01.793080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff81d3-1c74-4624-bafd-d6fd2aa30aee None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:01.793520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff81d3-1c74-4624-bafd-d6fd2aa30aee None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:01.794081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff81d3-1c74-4624-bafd-d6fd2aa30aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:01.816034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5fff81d3-1c74-4624-bafd-d6fd2aa30aee None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:01.817339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5fff81d3-1c74-4624-bafd-d6fd2aa30aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:01.831999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5329a2e-16ed-4e85-9942-38deabad07de admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:01.874959 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:01.911752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719355f9-f925-4220-9da6-a5d5bab41b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:01.912459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719355f9-f925-4220-9da6-a5d5bab41b3f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:01.912730 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719355f9-f925-4220-9da6-a5d5bab41b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:01.913006 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719355f9-f925-4220-9da6-a5d5bab41b3f admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:01.913764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719355f9-f925-4220-9da6-a5d5bab41b3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:01.915765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719355f9-f925-4220-9da6-a5d5bab41b3f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:01.931526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1018/2032] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:05:01.941701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1015/2033] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:05:01 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:05:01.946537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719355f9-f925-4220-9da6-a5d5bab41b3f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:01.947206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff81d3-1c74-4624-bafd-d6fd2aa30aee None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:01.948489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-719355f9-f925-4220-9da6-a5d5bab41b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:01.949545 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5fff81d3-1c74-4624-bafd-d6fd2aa30aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:01.961331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719355f9-f925-4220-9da6-a5d5bab41b3f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:01.966341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019b1d5-ee8f-4a2e-bbb2-ff3e5eac12e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:01.967094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019b1d5-ee8f-4a2e-bbb2-ff3e5eac12e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:01.967390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019b1d5-ee8f-4a2e-bbb2-ff3e5eac12e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:01.967664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019b1d5-ee8f-4a2e-bbb2-ff3e5eac12e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:01.968415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019b1d5-ee8f-4a2e-bbb2-ff3e5eac12e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:01.970108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019b1d5-ee8f-4a2e-bbb2-ff3e5eac12e1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:01.982758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1016/2034] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:01.987794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019b1d5-ee8f-4a2e-bbb2-ff3e5eac12e1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:01.998402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019b1d5-ee8f-4a2e-bbb2-ff3e5eac12e1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:02.003644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe500d5-e7be-4ca4-a81f-60dde7ffe93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:02.004386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe500d5-e7be-4ca4-a81f-60dde7ffe93a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:02.004686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe500d5-e7be-4ca4-a81f-60dde7ffe93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:02.004945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe500d5-e7be-4ca4-a81f-60dde7ffe93a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:02.005758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe500d5-e7be-4ca4-a81f-60dde7ffe93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:02.007418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe500d5-e7be-4ca4-a81f-60dde7ffe93a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:02.020706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1017/2035] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:02.025312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe500d5-e7be-4ca4-a81f-60dde7ffe93a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:02.038056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe500d5-e7be-4ca4-a81f-60dde7ffe93a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:02.044341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94527011-6d26-4026-9e9c-1e559bc01d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:02.045357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94527011-6d26-4026-9e9c-1e559bc01d1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:02.045678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94527011-6d26-4026-9e9c-1e559bc01d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:02.045962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94527011-6d26-4026-9e9c-1e559bc01d1a admin admin] PATH_INFO: `/v3/projects/ed4cebafbf2144d0b8e03b1756d502cf/users/81fa0383705d468484518c834bb8e20f/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:02.052720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:02.071752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94527011-6d26-4026-9e9c-1e559bc01d1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4cebafbf2144d0b8e03b1756d502cf, user_id=81fa0383705d468484518c834bb8e20f, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:02.075103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94527011-6d26-4026-9e9c-1e559bc01d1a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:02.085709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff81d3-1c74-4624-bafd-d6fd2aa30aee None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:02.093257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7b094-f1ea-4ac9-b2dd-b989c4147d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:02.094431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7b094-f1ea-4ac9-b2dd-b989c4147d8d service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:02.094917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7b094-f1ea-4ac9-b2dd-b989c4147d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:02.095460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7b094-f1ea-4ac9-b2dd-b989c4147d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:02.101328 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1018/2036] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:02 2022] PUT /identity/v3/projects/ed4cebafbf2144d0b8e03b1756d502cf/users/81fa0383705d468484518c834bb8e20f/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:05:02.108159 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7b094-f1ea-4ac9-b2dd-b989c4147d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:02.110245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7b094-f1ea-4ac9-b2dd-b989c4147d8d service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:02.123014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1019/2037] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:02.123292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4949-eac9-48a4-8fe8-914b4752f818 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:02.123869 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4949-eac9-48a4-8fe8-914b4752f818 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:02.124676 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4949-eac9-48a4-8fe8-914b4752f818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:02.181018 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c5bb4949-eac9-48a4-8fe8-914b4752f818 None None] MFA Rules not processed for user `81fa0383705d468484518c834bb8e20f`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:02.181815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c5bb4949-eac9-48a4-8fe8-914b4752f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:02.292866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1019/2038] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:05:02.299020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7b094-f1ea-4ac9-b2dd-b989c4147d8d service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:02.314523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7b094-f1ea-4ac9-b2dd-b989c4147d8d service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:02.458166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903016a-9d19-4244-8c4f-7125adda9d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:02.458819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903016a-9d19-4244-8c4f-7125adda9d88 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:02.459105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903016a-9d19-4244-8c4f-7125adda9d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:02.459382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903016a-9d19-4244-8c4f-7125adda9d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:02.460156 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903016a-9d19-4244-8c4f-7125adda9d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:02.462550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903016a-9d19-4244-8c4f-7125adda9d88 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:02.508018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1020/2039] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:05:02 2022] POST /identity/v3/projects => generated 377 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:02.512817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb4949-eac9-48a4-8fe8-914b4752f818 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:02.528852 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb4949-eac9-48a4-8fe8-914b4752f818 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:02.625858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e2b96-e313-486e-9fed-14b3634d9919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:02.626570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e2b96-e313-486e-9fed-14b3634d9919 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:02.626857 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e2b96-e313-486e-9fed-14b3634d9919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:02.627492 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e2b96-e313-486e-9fed-14b3634d9919 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:02.628261 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e2b96-e313-486e-9fed-14b3634d9919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:02.630316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e2b96-e313-486e-9fed-14b3634d9919 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:02.664744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1020/2040] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:05:02 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:05:02.665542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903016a-9d19-4244-8c4f-7125adda9d88 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:02.667507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e2b96-e313-486e-9fed-14b3634d9919 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:02.668599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9903016a-9d19-4244-8c4f-7125adda9d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:02.690531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e2b96-e313-486e-9fed-14b3634d9919 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:02.695873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903016a-9d19-4244-8c4f-7125adda9d88 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:02.703656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b579930-73f5-4aa4-b0f4-262f00081b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:02.706148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b579930-73f5-4aa4-b0f4-262f00081b75 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:02.706702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b579930-73f5-4aa4-b0f4-262f00081b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:02.707162 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b579930-73f5-4aa4-b0f4-262f00081b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:02.709905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b579930-73f5-4aa4-b0f4-262f00081b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:02.713458 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:02.714437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:02.714723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:02.715194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] PATH_INFO: `/v3/projects/01e901d5c0f24344b41fceee0c9d1f7d` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:02.716560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01e901d5c0f24344b41fceee0c9d1f7d)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:02.718897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:02.720872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b579930-73f5-4aa4-b0f4-262f00081b75 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:02.750670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1021/2041] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:02.756139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b579930-73f5-4aa4-b0f4-262f00081b75 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:02.772294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b579930-73f5-4aa4-b0f4-262f00081b75 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:02.777202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa523a8a-4d1d-459f-ac37-5125a137c4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:02.777975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa523a8a-4d1d-459f-ac37-5125a137c4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:02.778264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa523a8a-4d1d-459f-ac37-5125a137c4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:02.778542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa523a8a-4d1d-459f-ac37-5125a137c4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:02.779292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa523a8a-4d1d-459f-ac37-5125a137c4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:02.781336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa523a8a-4d1d-459f-ac37-5125a137c4c5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:02.803170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1022/2042] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:05:02.804845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa523a8a-4d1d-459f-ac37-5125a137c4c5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:02.813815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01e901d5c0f24344b41fceee0c9d1f7d', 'request_id': 'req-45d2a20e-417f-40a7-aae6-01eda524bf38'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:02.829439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa523a8a-4d1d-459f-ac37-5125a137c4c5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:02.830861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01e901d5c0f24344b41fceee0c9d1f7d', 'request_id': 'req-45d2a20e-417f-40a7-aae6-01eda524bf38'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:02.851295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01e901d5c0f24344b41fceee0c9d1f7d', 'request_id': 'req-45d2a20e-417f-40a7-aae6-01eda524bf38'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:02.864200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:02.909978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] The token cache is being invalidate because project 01e901d5c0f24344b41fceee0c9d1f7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:02.910439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:02.911405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:02.915434 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:02.922482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1021/2043] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:02 2022] DELETE /identity/v3/projects/01e901d5c0f24344b41fceee0c9d1f7d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:02.923880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:02.931979 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:02.958606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470e89b-cf90-4c10-a575-9d2966845dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:02.961681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470e89b-cf90-4c10-a575-9d2966845dbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:02.970912 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470e89b-cf90-4c10-a575-9d2966845dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:02.971501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470e89b-cf90-4c10-a575-9d2966845dbd admin admin] PATH_INFO: `/v3/projects/af82ac7930ee49e8b26b0137aa1619a9/users/02c00e1e3c9440ff9b0601a58e9096cb/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:03.006544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470e89b-cf90-4c10-a575-9d2966845dbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=af82ac7930ee49e8b26b0137aa1619a9, user_id=02c00e1e3c9440ff9b0601a58e9096cb, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:03.009670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470e89b-cf90-4c10-a575-9d2966845dbd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:03.036866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1023/2044] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:02 2022] PUT /identity/v3/projects/af82ac7930ee49e8b26b0137aa1619a9/users/02c00e1e3c9440ff9b0601a58e9096cb/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:03.060280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582c94a-f84e-4d5a-b529-e626e971c932 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:03.060553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582c94a-f84e-4d5a-b529-e626e971c932 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:03.060874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582c94a-f84e-4d5a-b529-e626e971c932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:03.097207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2a20e-417f-40a7-aae6-01eda524bf38 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:03.136353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6582c94a-f84e-4d5a-b529-e626e971c932 None None] MFA Rules not processed for user `02c00e1e3c9440ff9b0601a58e9096cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:03.137677 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6582c94a-f84e-4d5a-b529-e626e971c932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:03.247734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:03.248988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:03.249276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:03.249653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] PATH_INFO: `/v3/users/83c2b077048d4ec5a818c7b2c062422c` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:03.252728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83c2b077048d4ec5a818c7b2c062422c)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:03.255878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:03.321533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1024/2045] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:03.328352 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582c94a-f84e-4d5a-b529-e626e971c932 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:03.330963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6582c94a-f84e-4d5a-b529-e626e971c932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:03.348608 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582c94a-f84e-4d5a-b529-e626e971c932 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:03.349368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83c2b077048d4ec5a818c7b2c062422c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:03.350389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83c2b077048d4ec5a818c7b2c062422c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:03.351370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83c2b077048d4ec5a818c7b2c062422c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:03.355029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23326013-e68c-4f9a-828d-c2a9242b58a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:03.355903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23326013-e68c-4f9a-828d-c2a9242b58a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:03.356285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23326013-e68c-4f9a-828d-c2a9242b58a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:03.356660 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23326013-e68c-4f9a-828d-c2a9242b58a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:03.358160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23326013-e68c-4f9a-828d-c2a9242b58a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:03.360731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23326013-e68c-4f9a-828d-c2a9242b58a5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:03.363221 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83c2b077048d4ec5a818c7b2c062422c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:03.379542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83c2b077048d4ec5a818c7b2c062422c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:03.399457 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1025/2046] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:05:03 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:03.403569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23326013-e68c-4f9a-828d-c2a9242b58a5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:03.405345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1022/2047] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:02 2022] DELETE /identity/v3/users/83c2b077048d4ec5a818c7b2c062422c => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:03.407235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-23326013-e68c-4f9a-828d-c2a9242b58a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:03.513035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23326013-e68c-4f9a-828d-c2a9242b58a5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:03.584616 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:03.635053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab15cb2-f6b4-4a7e-914d-c7078689e561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:03.635699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab15cb2-f6b4-4a7e-914d-c7078689e561 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:03.636126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab15cb2-f6b4-4a7e-914d-c7078689e561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:03.636609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab15cb2-f6b4-4a7e-914d-c7078689e561 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:03.637894 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab15cb2-f6b4-4a7e-914d-c7078689e561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:03.641333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab15cb2-f6b4-4a7e-914d-c7078689e561 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:03.688544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1026/2048] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:05:03 2022] POST /identity/v3/users => generated 295 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:03.689453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:03.717247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91756a38-8d4b-440c-a935-da15038759c0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:03.724571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d9b87-3f0f-4810-9cf6-a2793aae6c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:03.725291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d9b87-3f0f-4810-9cf6-a2793aae6c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:03.725688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d9b87-3f0f-4810-9cf6-a2793aae6c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:03.726035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d9b87-3f0f-4810-9cf6-a2793aae6c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:03.726794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d9b87-3f0f-4810-9cf6-a2793aae6c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:03.728523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d9b87-3f0f-4810-9cf6-a2793aae6c5e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:03.745732 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1023/2049] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:03.750628 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab15cb2-f6b4-4a7e-914d-c7078689e561 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:03.770919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab15cb2-f6b4-4a7e-914d-c7078689e561 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:03.779940 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2beb8a-281c-403d-a68e-cbb2adce6ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:03.781689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2beb8a-281c-403d-a68e-cbb2adce6ec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:03.782219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2beb8a-281c-403d-a68e-cbb2adce6ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:03.782772 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2beb8a-281c-403d-a68e-cbb2adce6ec6 admin admin] PATH_INFO: `/v3/projects/340e27bbd3f143ad908b3543cc25bc8a/users/d194c9cb0bc946c0a057b2c4653d53fe/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:03.819126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2beb8a-281c-403d-a68e-cbb2adce6ec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=340e27bbd3f143ad908b3543cc25bc8a, user_id=d194c9cb0bc946c0a057b2c4653d53fe, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:03.830125 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2beb8a-281c-403d-a68e-cbb2adce6ec6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:03.862543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1027/2050] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:03 2022] PUT /identity/v3/projects/340e27bbd3f143ad908b3543cc25bc8a/users/d194c9cb0bc946c0a057b2c4653d53fe/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:03.865280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d9b87-3f0f-4810-9cf6-a2793aae6c5e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:03.881841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d9b87-3f0f-4810-9cf6-a2793aae6c5e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:03.905366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:03.929216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2beb8a-281c-403d-a68e-cbb2adce6ec6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:03.934510 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4a2beb8a-281c-403d-a68e-cbb2adce6ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:03.997155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e1f8e-52ef-4dd5-8419-1c9e9e21ff50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:03.997892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e1f8e-52ef-4dd5-8419-1c9e9e21ff50 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:03.998176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e1f8e-52ef-4dd5-8419-1c9e9e21ff50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:03.998450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e1f8e-52ef-4dd5-8419-1c9e9e21ff50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:03.999213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e1f8e-52ef-4dd5-8419-1c9e9e21ff50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:04.001018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e1f8e-52ef-4dd5-8419-1c9e9e21ff50 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:04.019161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1024/2051] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:04.022817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e1f8e-52ef-4dd5-8419-1c9e9e21ff50 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:04.039397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e1f8e-52ef-4dd5-8419-1c9e9e21ff50 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:04.045171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6173d2-b4cd-4276-91bd-74a7a09be274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:04.046150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6173d2-b4cd-4276-91bd-74a7a09be274 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:04.046442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6173d2-b4cd-4276-91bd-74a7a09be274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:04.046788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6173d2-b4cd-4276-91bd-74a7a09be274 admin admin] PATH_INFO: `/v3/projects/340e27bbd3f143ad908b3543cc25bc8a/users/d194c9cb0bc946c0a057b2c4653d53fe/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:04.049366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6173d2-b4cd-4276-91bd-74a7a09be274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=340e27bbd3f143ad908b3543cc25bc8a, user_id=d194c9cb0bc946c0a057b2c4653d53fe, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:04.052309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6173d2-b4cd-4276-91bd-74a7a09be274 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:04.086466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1025/2052] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:04 2022] PUT /identity/v3/projects/340e27bbd3f143ad908b3543cc25bc8a/users/d194c9cb0bc946c0a057b2c4653d53fe/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:05:04.087339 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2beb8a-281c-403d-a68e-cbb2adce6ec6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:04.093615 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaea3fc-e45b-403c-853d-5f9ca2b328d4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:04.094470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaea3fc-e45b-403c-853d-5f9ca2b328d4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:04.095400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaea3fc-e45b-403c-853d-5f9ca2b328d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:04.138318 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-aeaea3fc-e45b-403c-853d-5f9ca2b328d4 None None] MFA Rules not processed for user `d194c9cb0bc946c0a057b2c4653d53fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:04.139630 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aeaea3fc-e45b-403c-853d-5f9ca2b328d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:04.179805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c356ae-6f70-4346-994e-40620c630cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:04.181895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c356ae-6f70-4346-994e-40620c630cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:04.182305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c356ae-6f70-4346-994e-40620c630cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:04.182680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c356ae-6f70-4346-994e-40620c630cf2 admin admin] PATH_INFO: `/v3/projects/35e2d940ed954d109e7eec97f5bd4dbc` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:04.185515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c356ae-6f70-4346-994e-40620c630cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e2d940ed954d109e7eec97f5bd4dbc)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:04.188801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c356ae-6f70-4346-994e-40620c630cf2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:04.230846 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1026/2053] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:04.245243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaea3fc-e45b-403c-853d-5f9ca2b328d4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:04.247030 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aeaea3fc-e45b-403c-853d-5f9ca2b328d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:04.319369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-95c356ae-6f70-4346-994e-40620c630cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e2d940ed954d109e7eec97f5bd4dbc', 'request_id': 'req-95c356ae-6f70-4346-994e-40620c630cf2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:04.324614 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:04.339599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-95c356ae-6f70-4346-994e-40620c630cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e2d940ed954d109e7eec97f5bd4dbc', 'request_id': 'req-95c356ae-6f70-4346-994e-40620c630cf2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:04.358367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-95c356ae-6f70-4346-994e-40620c630cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e2d940ed954d109e7eec97f5bd4dbc', 'request_id': 'req-95c356ae-6f70-4346-994e-40620c630cf2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:04.364989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaea3fc-e45b-403c-853d-5f9ca2b328d4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:04.386758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-95c356ae-6f70-4346-994e-40620c630cf2 admin admin] The token cache is being invalidate because project 35e2d940ed954d109e7eec97f5bd4dbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:04.387061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-95c356ae-6f70-4346-994e-40620c630cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:04.390829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-95c356ae-6f70-4346-994e-40620c630cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:04.393922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1028/2054] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:03 2022] DELETE /identity/v3/projects/35e2d940ed954d109e7eec97f5bd4dbc => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:05:04.399915 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574eb095-fa48-45c5-a6f4-d954d637aee3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:04.400781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574eb095-fa48-45c5-a6f4-d954d637aee3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:04.401265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574eb095-fa48-45c5-a6f4-d954d637aee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:04.430091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:04.453907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-574eb095-fa48-45c5-a6f4-d954d637aee3 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:04.455227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-574eb095-fa48-45c5-a6f4-d954d637aee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:04.474495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d938c-7f6d-4f71-8a00-063bcf80132d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:04.475181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d938c-7f6d-4f71-8a00-063bcf80132d service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:04.475462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d938c-7f6d-4f71-8a00-063bcf80132d service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:04.475888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d938c-7f6d-4f71-8a00-063bcf80132d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:04.478979 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7f8d938c-7f6d-4f71-8a00-063bcf80132d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:04.625369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d938c-7f6d-4f71-8a00-063bcf80132d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:04.628184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d938c-7f6d-4f71-8a00-063bcf80132d service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:04.631969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1029/2055] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:05:04.646741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089a8a1-ce5a-4a2c-8c40-d818b09a8e38 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:04.647144 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089a8a1-ce5a-4a2c-8c40-d818b09a8e38 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:04.647597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089a8a1-ce5a-4a2c-8c40-d818b09a8e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:04.668138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1027/2056] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:04.726947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e089a8a1-ce5a-4a2c-8c40-d818b09a8e38 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:04.729607 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e089a8a1-ce5a-4a2c-8c40-d818b09a8e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:04.770472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d938c-7f6d-4f71-8a00-063bcf80132d service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:04.775064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1030/2057] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:04.793580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d938c-7f6d-4f71-8a00-063bcf80132d service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:04.809708 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5801bc-568b-4b0b-ba61-25484b088185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:04.810613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5801bc-568b-4b0b-ba61-25484b088185 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:04.811194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5801bc-568b-4b0b-ba61-25484b088185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:04.811472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5801bc-568b-4b0b-ba61-25484b088185 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:04.813179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5801bc-568b-4b0b-ba61-25484b088185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:05:04.813502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5801bc-568b-4b0b-ba61-25484b088185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:04.816114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5801bc-568b-4b0b-ba61-25484b088185 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:04.851619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1028/2058] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:05:04.857312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089a8a1-ce5a-4a2c-8c40-d818b09a8e38 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:04.874546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089a8a1-ce5a-4a2c-8c40-d818b09a8e38 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:04.885694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a75b18-8f54-43e4-8735-14c2881836cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:04.886986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a75b18-8f54-43e4-8735-14c2881836cf admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:04.887148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a75b18-8f54-43e4-8735-14c2881836cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:04.887410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a75b18-8f54-43e4-8735-14c2881836cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:04.889147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a75b18-8f54-43e4-8735-14c2881836cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:04.891520 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a75b18-8f54-43e4-8735-14c2881836cf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:04.952744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1031/2059] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:05:04 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:04.957871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5801bc-568b-4b0b-ba61-25484b088185 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:04.978908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5801bc-568b-4b0b-ba61-25484b088185 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:05.154168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c140d-fbf5-48c7-8d90-3f50d45f0b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:05.155209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c140d-fbf5-48c7-8d90-3f50d45f0b68 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:05.155600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c140d-fbf5-48c7-8d90-3f50d45f0b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:05.156034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c140d-fbf5-48c7-8d90-3f50d45f0b68 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:05.157252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c140d-fbf5-48c7-8d90-3f50d45f0b68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:05.166115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c140d-fbf5-48c7-8d90-3f50d45f0b68 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:05.206423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a75b18-8f54-43e4-8735-14c2881836cf admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:05.206913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1029/2060] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:05:04 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:05.228061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a75b18-8f54-43e4-8735-14c2881836cf admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:05.237707 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64e8e8-b684-4fa8-8377-b6df4fc3f217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:05.239718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64e8e8-b684-4fa8-8377-b6df4fc3f217 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:05.240442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64e8e8-b684-4fa8-8377-b6df4fc3f217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:05.240951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64e8e8-b684-4fa8-8377-b6df4fc3f217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:05.241971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64e8e8-b684-4fa8-8377-b6df4fc3f217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:05.243995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64e8e8-b684-4fa8-8377-b6df4fc3f217 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:05.262802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1032/2061] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:05.267272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c140d-fbf5-48c7-8d90-3f50d45f0b68 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:05.281090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c140d-fbf5-48c7-8d90-3f50d45f0b68 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:05.287393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02efa4-3340-4c9e-8bad-07f9e1b77316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:05.288113 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02efa4-3340-4c9e-8bad-07f9e1b77316 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:05.288484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02efa4-3340-4c9e-8bad-07f9e1b77316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:05.288761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02efa4-3340-4c9e-8bad-07f9e1b77316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:05.289789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02efa4-3340-4c9e-8bad-07f9e1b77316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:05.291818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02efa4-3340-4c9e-8bad-07f9e1b77316 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:05.305826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1030/2062] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:05.309436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64e8e8-b684-4fa8-8377-b6df4fc3f217 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:05.325201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64e8e8-b684-4fa8-8377-b6df4fc3f217 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:05.332809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44905c9-78e9-41a8-adaa-84c22f50fe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:05.334166 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44905c9-78e9-41a8-adaa-84c22f50fe03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:05.334520 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44905c9-78e9-41a8-adaa-84c22f50fe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:05.334743 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44905c9-78e9-41a8-adaa-84c22f50fe03 admin admin] PATH_INFO: `/v3/projects/6cdd4e070c2843a0b86db6712855d12f/users/3bf6e92f88954922a0507f82681396b8/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:05.359079 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44905c9-78e9-41a8-adaa-84c22f50fe03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cdd4e070c2843a0b86db6712855d12f, user_id=3bf6e92f88954922a0507f82681396b8, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:05.363247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44905c9-78e9-41a8-adaa-84c22f50fe03 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:05.381356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1033/2063] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:05 2022] PUT /identity/v3/projects/6cdd4e070c2843a0b86db6712855d12f/users/3bf6e92f88954922a0507f82681396b8/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:05:05.395847 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d7598-57ee-476c-af67-e341c85def04 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:05.396146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d7598-57ee-476c-af67-e341c85def04 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:05.396398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d7598-57ee-476c-af67-e341c85def04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:05.437896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b60d7598-57ee-476c-af67-e341c85def04 None None] MFA Rules not processed for user `3bf6e92f88954922a0507f82681396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:05.441851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b60d7598-57ee-476c-af67-e341c85def04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:05.531508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:05.585355 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1031/2064] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:05.592727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44905c9-78e9-41a8-adaa-84c22f50fe03 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:05.694437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44905c9-78e9-41a8-adaa-84c22f50fe03 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:05.703105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e022b-b0cb-41bc-b485-b4d02bbd879f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:05.704807 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e022b-b0cb-41bc-b485-b4d02bbd879f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:05.705731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e022b-b0cb-41bc-b485-b4d02bbd879f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:05.706514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e022b-b0cb-41bc-b485-b4d02bbd879f admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:05.708200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e022b-b0cb-41bc-b485-b4d02bbd879f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:05.712540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e022b-b0cb-41bc-b485-b4d02bbd879f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:05.778040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1034/2065] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:05:05 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:05:05.779451 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d7598-57ee-476c-af67-e341c85def04 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:05.802214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d7598-57ee-476c-af67-e341c85def04 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:05.933144 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffc3ab-e688-4b95-bf72-05ccfbe3d773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:05.934025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffc3ab-e688-4b95-bf72-05ccfbe3d773 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:05.934392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffc3ab-e688-4b95-bf72-05ccfbe3d773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:05.934763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffc3ab-e688-4b95-bf72-05ccfbe3d773 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:05.935571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffc3ab-e688-4b95-bf72-05ccfbe3d773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:05.937683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffc3ab-e688-4b95-bf72-05ccfbe3d773 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:05.984247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1032/2066] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:05:05 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:05:05.986212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e022b-b0cb-41bc-b485-b4d02bbd879f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:05.996059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:06.009770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e022b-b0cb-41bc-b485-b4d02bbd879f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:06.019274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af4f36-cf3a-4f87-9f09-ee67ebbebf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:06.020578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af4f36-cf3a-4f87-9f09-ee67ebbebf57 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:06.021038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af4f36-cf3a-4f87-9f09-ee67ebbebf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:06.021477 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af4f36-cf3a-4f87-9f09-ee67ebbebf57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:06.031145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af4f36-cf3a-4f87-9f09-ee67ebbebf57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:06.034194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af4f36-cf3a-4f87-9f09-ee67ebbebf57 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:06.063530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1035/2067] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:06.065409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffc3ab-e688-4b95-bf72-05ccfbe3d773 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:06.085304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffc3ab-e688-4b95-bf72-05ccfbe3d773 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:06.098286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccea7d3-47fd-41db-ad80-a522e3943390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:06.099921 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccea7d3-47fd-41db-ad80-a522e3943390 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:06.100698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccea7d3-47fd-41db-ad80-a522e3943390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:06.101263 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccea7d3-47fd-41db-ad80-a522e3943390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:06.129401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccea7d3-47fd-41db-ad80-a522e3943390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:06.136922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccea7d3-47fd-41db-ad80-a522e3943390 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:06.165749 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1033/2068] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:06.172638 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af4f36-cf3a-4f87-9f09-ee67ebbebf57 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:06.238927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af4f36-cf3a-4f87-9f09-ee67ebbebf57 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:06.238927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02141a4-320a-4399-b537-cdaba1126d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:06.238927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02141a4-320a-4399-b537-cdaba1126d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:06.238927 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02141a4-320a-4399-b537-cdaba1126d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:06.240316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02141a4-320a-4399-b537-cdaba1126d35 admin admin] PATH_INFO: `/v3/projects/7662310a43b845269a2d390a8091f5f1/users/0bc11651fe7b44e79a1bc94ae36857c5/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:06.291749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02141a4-320a-4399-b537-cdaba1126d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7662310a43b845269a2d390a8091f5f1, user_id=0bc11651fe7b44e79a1bc94ae36857c5, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:06.302865 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02141a4-320a-4399-b537-cdaba1126d35 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:06.329729 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1036/2069] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:06 2022] PUT /identity/v3/projects/7662310a43b845269a2d390a8091f5f1/users/0bc11651fe7b44e79a1bc94ae36857c5/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:06.352667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e1e4b-ee52-4fc8-95ed-99959b46cc7d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:06.353202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e1e4b-ee52-4fc8-95ed-99959b46cc7d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:06.353978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e1e4b-ee52-4fc8-95ed-99959b46cc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:06.421933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d88e1e4b-ee52-4fc8-95ed-99959b46cc7d None None] MFA Rules not processed for user `0bc11651fe7b44e79a1bc94ae36857c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:06.424120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d88e1e4b-ee52-4fc8-95ed-99959b46cc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:06.516470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1034/2070] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:06.521829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02141a4-320a-4399-b537-cdaba1126d35 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:06.613937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02141a4-320a-4399-b537-cdaba1126d35 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:06.628250 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85b1cb-ff43-4c1f-be7c-81b1f4762d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:06.629352 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85b1cb-ff43-4c1f-be7c-81b1f4762d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:06.629851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85b1cb-ff43-4c1f-be7c-81b1f4762d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:06.630609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85b1cb-ff43-4c1f-be7c-81b1f4762d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:06.632042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85b1cb-ff43-4c1f-be7c-81b1f4762d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:06.635640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85b1cb-ff43-4c1f-be7c-81b1f4762d60 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:06.683917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1037/2071] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:05:06 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:06.685330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e1e4b-ee52-4fc8-95ed-99959b46cc7d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:06.687702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29218bd-4027-4517-949c-26f0308e0cd8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:06.688253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29218bd-4027-4517-949c-26f0308e0cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:06.688732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29218bd-4027-4517-949c-26f0308e0cd8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:06.691662 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1038/2072] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 08:05:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:05:06.699039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e1e4b-ee52-4fc8-95ed-99959b46cc7d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:06.704290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a2061-af1c-4486-9e06-4c58caa5b069 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:06.704871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a2061-af1c-4486-9e06-4c58caa5b069 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:06.709301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a2061-af1c-4486-9e06-4c58caa5b069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:06.747897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1f5a2061-af1c-4486-9e06-4c58caa5b069 None None] MFA Rules not processed for user `f57dccfe9f5f4bc397f4c470fbf047dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:06.751030 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1f5a2061-af1c-4486-9e06-4c58caa5b069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:06.785731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:06.807668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554155fa-5a8a-4fca-b1ab-f25e5ee2db23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:06.808412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554155fa-5a8a-4fca-b1ab-f25e5ee2db23 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:06.808665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554155fa-5a8a-4fca-b1ab-f25e5ee2db23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:06.809128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554155fa-5a8a-4fca-b1ab-f25e5ee2db23 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:06.810064 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554155fa-5a8a-4fca-b1ab-f25e5ee2db23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:06.812187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554155fa-5a8a-4fca-b1ab-f25e5ee2db23 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:06.847372 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1035/2073] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:05:06 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:06.847372 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554155fa-5a8a-4fca-b1ab-f25e5ee2db23 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:06.848524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-554155fa-5a8a-4fca-b1ab-f25e5ee2db23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:06.862554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1039/2074] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:06.863575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a2061-af1c-4486-9e06-4c58caa5b069 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:06.871395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554155fa-5a8a-4fca-b1ab-f25e5ee2db23 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:06.876967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a2061-af1c-4486-9e06-4c58caa5b069 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:06.879567 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:06.881137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:06.881584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:06.882121 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] PATH_INFO: `/v3/users/31fe6ac873124636977d593ad5dc2b65` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:06.883582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3e0e1-5793-4deb-8475-b0575578a30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:06.884225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=31fe6ac873124636977d593ad5dc2b65)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:06.884320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3e0e1-5793-4deb-8475-b0575578a30e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:06.884716 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3e0e1-5793-4deb-8475-b0575578a30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:06.885000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3e0e1-5793-4deb-8475-b0575578a30e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:06.885781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3e0e1-5793-4deb-8475-b0575578a30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:06.887695 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:06.888163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3e0e1-5793-4deb-8475-b0575578a30e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:06.903631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1040/2075] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:06.903631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3e0e1-5793-4deb-8475-b0575578a30e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:06.903631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-90e3e0e1-5793-4deb-8475-b0575578a30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:06.919550 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3e0e1-5793-4deb-8475-b0575578a30e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:06.926301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75e906-291a-4867-8e10-8fd6e859c2dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57dccfe9f5f4bc397f4c470fbf047dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:06.926889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75e906-291a-4867-8e10-8fd6e859c2dc service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:06.927218 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75e906-291a-4867-8e10-8fd6e859c2dc service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:06.927586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75e906-291a-4867-8e10-8fd6e859c2dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:06.938641 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75e906-291a-4867-8e10-8fd6e859c2dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:06.940394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75e906-291a-4867-8e10-8fd6e859c2dc service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:06.955480 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1041/2076] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:06.957980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75e906-291a-4867-8e10-8fd6e859c2dc service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:06.966176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31fe6ac873124636977d593ad5dc2b65'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:06.966930 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31fe6ac873124636977d593ad5dc2b65'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:06.967579 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31fe6ac873124636977d593ad5dc2b65'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:06.978516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75e906-291a-4867-8e10-8fd6e859c2dc service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:06.980301 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31fe6ac873124636977d593ad5dc2b65'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:06.987091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b85dbb-e8e8-406a-8271-3167abab2cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:06.988411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b85dbb-e8e8-406a-8271-3167abab2cec admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:06.989231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b85dbb-e8e8-406a-8271-3167abab2cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:06.989231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b85dbb-e8e8-406a-8271-3167abab2cec admin admin] PATH_INFO: `/v3/projects/43cfbe79e8fa4903994fb79e045a3bae/users/2db5d8a156994821a4fb0a14f83a2665/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:06.998182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31fe6ac873124636977d593ad5dc2b65'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:07.008070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b85dbb-e8e8-406a-8271-3167abab2cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=43cfbe79e8fa4903994fb79e045a3bae, user_id=2db5d8a156994821a4fb0a14f83a2665, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:07.010965 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b85dbb-e8e8-406a-8271-3167abab2cec admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:07.015951 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1036/2077] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:06 2022] DELETE /identity/v3/users/31fe6ac873124636977d593ad5dc2b65 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:07.023014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:07.030906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1042/2078] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:06 2022] PUT /identity/v3/projects/43cfbe79e8fa4903994fb79e045a3bae/users/2db5d8a156994821a4fb0a14f83a2665/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:07.036126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:07.038235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:07.124450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf07e4-5df3-4daf-9796-39ee684ab4ff admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:07.193186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183af90-695b-4aec-87b1-dddd49319861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:07.194254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183af90-695b-4aec-87b1-dddd49319861 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:07.194254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183af90-695b-4aec-87b1-dddd49319861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:07.194674 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183af90-695b-4aec-87b1-dddd49319861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:07.195210 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183af90-695b-4aec-87b1-dddd49319861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:07.196864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183af90-695b-4aec-87b1-dddd49319861 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:07.198521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:07.214632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1037/2079] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:07.216054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b85dbb-e8e8-406a-8271-3167abab2cec admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:07.230394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b85dbb-e8e8-406a-8271-3167abab2cec admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:07.235630 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183af90-695b-4aec-87b1-dddd49319861 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:07.236325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41503786-af3f-482b-836b-0ed90bc569cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:07.237674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41503786-af3f-482b-836b-0ed90bc569cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:07.238132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41503786-af3f-482b-836b-0ed90bc569cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:07.238429 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41503786-af3f-482b-836b-0ed90bc569cb admin admin] PATH_INFO: `/v3/projects/43cfbe79e8fa4903994fb79e045a3bae/users/2db5d8a156994821a4fb0a14f83a2665/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:07.240145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a183af90-695b-4aec-87b1-dddd49319861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:07.240734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41503786-af3f-482b-836b-0ed90bc569cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=43cfbe79e8fa4903994fb79e045a3bae, user_id=2db5d8a156994821a4fb0a14f83a2665, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:07.243979 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41503786-af3f-482b-836b-0ed90bc569cb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:07.260825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1043/2080] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:07 2022] PUT /identity/v3/projects/43cfbe79e8fa4903994fb79e045a3bae/users/2db5d8a156994821a4fb0a14f83a2665/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:05:07.267887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183af90-695b-4aec-87b1-dddd49319861 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:07.271803 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702a36c-cdbb-4268-bcc8-d43d9b893f9e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:07.272411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702a36c-cdbb-4268-bcc8-d43d9b893f9e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:07.272923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702a36c-cdbb-4268-bcc8-d43d9b893f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:07.312029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8702a36c-cdbb-4268-bcc8-d43d9b893f9e None None] MFA Rules not processed for user `2db5d8a156994821a4fb0a14f83a2665`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:07.313316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8702a36c-cdbb-4268-bcc8-d43d9b893f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:07.380474 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e18c36-b816-44eb-a7a2-fcbebde0ff17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:07.382242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e18c36-b816-44eb-a7a2-fcbebde0ff17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:07.382778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e18c36-b816-44eb-a7a2-fcbebde0ff17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:07.383311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e18c36-b816-44eb-a7a2-fcbebde0ff17 admin admin] PATH_INFO: `/v3/projects/7de2b98bf8c340d0b7646468c3594a47` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:07.385374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e18c36-b816-44eb-a7a2-fcbebde0ff17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7de2b98bf8c340d0b7646468c3594a47)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:07.389017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e18c36-b816-44eb-a7a2-fcbebde0ff17 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:07.417657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1044/2081] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:07.434312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702a36c-cdbb-4268-bcc8-d43d9b893f9e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:07.436112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8702a36c-cdbb-4268-bcc8-d43d9b893f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:07.456365 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-21e18c36-b816-44eb-a7a2-fcbebde0ff17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7de2b98bf8c340d0b7646468c3594a47', 'request_id': 'req-21e18c36-b816-44eb-a7a2-fcbebde0ff17'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:07.463603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-21e18c36-b816-44eb-a7a2-fcbebde0ff17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7de2b98bf8c340d0b7646468c3594a47', 'request_id': 'req-21e18c36-b816-44eb-a7a2-fcbebde0ff17'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:07.471634 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-21e18c36-b816-44eb-a7a2-fcbebde0ff17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7de2b98bf8c340d0b7646468c3594a47', 'request_id': 'req-21e18c36-b816-44eb-a7a2-fcbebde0ff17'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:07.487673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-21e18c36-b816-44eb-a7a2-fcbebde0ff17 admin admin] The token cache is being invalidate because project 7de2b98bf8c340d0b7646468c3594a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:07.488155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-21e18c36-b816-44eb-a7a2-fcbebde0ff17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:07.489066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-21e18c36-b816-44eb-a7a2-fcbebde0ff17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:07.491733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1038/2082] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:07 2022] DELETE /identity/v3/projects/7de2b98bf8c340d0b7646468c3594a47 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:07.497346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51875a1e-e46f-4427-b1cd-7aadd245f7a3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:07.497614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51875a1e-e46f-4427-b1cd-7aadd245f7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:07.497943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51875a1e-e46f-4427-b1cd-7aadd245f7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:07.510367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702a36c-cdbb-4268-bcc8-d43d9b893f9e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:07.520687 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-51875a1e-e46f-4427-b1cd-7aadd245f7a3 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:07.522039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-51875a1e-e46f-4427-b1cd-7aadd245f7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:07.569803 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16b528-a314-4f49-8a7a-1f2b4e3df4cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:07.570674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16b528-a314-4f49-8a7a-1f2b4e3df4cd service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:07.571078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16b528-a314-4f49-8a7a-1f2b4e3df4cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:07.571431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16b528-a314-4f49-8a7a-1f2b4e3df4cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:07.573916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0c16b528-a314-4f49-8a7a-1f2b4e3df4cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:07.602646 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1039/2083] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:05:07.611912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0643d4e-9539-4c88-90dc-514983d38b7b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:07.612199 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0643d4e-9539-4c88-90dc-514983d38b7b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:07.612492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0643d4e-9539-4c88-90dc-514983d38b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:07.635423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:07.641856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d0643d4e-9539-4c88-90dc-514983d38b7b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:07.644134 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d0643d4e-9539-4c88-90dc-514983d38b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:07.655588 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16b528-a314-4f49-8a7a-1f2b4e3df4cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:07.657065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16b528-a314-4f49-8a7a-1f2b4e3df4cd service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:07.659370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1040/2084] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:05:07.664295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0643d4e-9539-4c88-90dc-514983d38b7b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:07.668611 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1045/2085] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:05:07.675286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0643d4e-9539-4c88-90dc-514983d38b7b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:07.680729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04026a6-e028-42ce-b562-bc15b6a4d579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:07.681428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04026a6-e028-42ce-b562-bc15b6a4d579 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:07.681763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04026a6-e028-42ce-b562-bc15b6a4d579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:07.682046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04026a6-e028-42ce-b562-bc15b6a4d579 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:07.682810 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04026a6-e028-42ce-b562-bc15b6a4d579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:05:07.683160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04026a6-e028-42ce-b562-bc15b6a4d579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:07.684799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04026a6-e028-42ce-b562-bc15b6a4d579 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:07.705372 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1041/2086] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:05:07.707266 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16b528-a314-4f49-8a7a-1f2b4e3df4cd service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:07.718756 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16b528-a314-4f49-8a7a-1f2b4e3df4cd service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:07.726736 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf179a17-9090-4716-b5ec-8223cd6a4b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:07.727649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf179a17-9090-4716-b5ec-8223cd6a4b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:07.728239 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf179a17-9090-4716-b5ec-8223cd6a4b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:07.728762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf179a17-9090-4716-b5ec-8223cd6a4b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:07.729590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf179a17-9090-4716-b5ec-8223cd6a4b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:07.732026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf179a17-9090-4716-b5ec-8223cd6a4b9a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:07.772866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1046/2087] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:05:07 2022] POST /identity/v3/projects => generated 391 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:07.777827 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04026a6-e028-42ce-b562-bc15b6a4d579 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:07.796786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04026a6-e028-42ce-b562-bc15b6a4d579 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:07.803720 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:07.891232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209107ac-b9f0-4c8f-a484-e74ba6f37fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:07.892442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209107ac-b9f0-4c8f-a484-e74ba6f37fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:07.893000 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209107ac-b9f0-4c8f-a484-e74ba6f37fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:07.893523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209107ac-b9f0-4c8f-a484-e74ba6f37fb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:07.894922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209107ac-b9f0-4c8f-a484-e74ba6f37fb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:07.899006 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209107ac-b9f0-4c8f-a484-e74ba6f37fb5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:07.933350 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1042/2088] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:05:07 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:07.940279 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf179a17-9090-4716-b5ec-8223cd6a4b9a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:07.961211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf179a17-9090-4716-b5ec-8223cd6a4b9a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:07.969326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e32f43-c770-4aed-91f9-1b4714aa2563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:07.970471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e32f43-c770-4aed-91f9-1b4714aa2563 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:07.970925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e32f43-c770-4aed-91f9-1b4714aa2563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:07.971569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e32f43-c770-4aed-91f9-1b4714aa2563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:07.972945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e32f43-c770-4aed-91f9-1b4714aa2563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:07.975829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e32f43-c770-4aed-91f9-1b4714aa2563 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:07.995694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1047/2089] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:08.000716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209107ac-b9f0-4c8f-a484-e74ba6f37fb5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:08.026028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209107ac-b9f0-4c8f-a484-e74ba6f37fb5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:08.035886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d01c4a-8337-49b9-b30c-5a07bc8a3b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:08.037373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d01c4a-8337-49b9-b30c-5a07bc8a3b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:08.038726 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d01c4a-8337-49b9-b30c-5a07bc8a3b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:08.038832 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d01c4a-8337-49b9-b30c-5a07bc8a3b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:08.040407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d01c4a-8337-49b9-b30c-5a07bc8a3b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:08.043549 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d01c4a-8337-49b9-b30c-5a07bc8a3b09 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:08.074323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1043/2090] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:05:08.083650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e32f43-c770-4aed-91f9-1b4714aa2563 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:08.103569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e32f43-c770-4aed-91f9-1b4714aa2563 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:08.112657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de0b1a-1c3f-4167-90a9-6763d943387a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:08.115060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de0b1a-1c3f-4167-90a9-6763d943387a admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:08.115060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de0b1a-1c3f-4167-90a9-6763d943387a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:08.115418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de0b1a-1c3f-4167-90a9-6763d943387a admin admin] PATH_INFO: `/v3/projects/ecd5bc804a3b4f14b864fb4e0f8d1b98/users/9e3f73e469f34b66bfbe99c3d79ac129/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:08.146250 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de0b1a-1c3f-4167-90a9-6763d943387a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd5bc804a3b4f14b864fb4e0f8d1b98, user_id=9e3f73e469f34b66bfbe99c3d79ac129, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:08.150441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de0b1a-1c3f-4167-90a9-6763d943387a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:08.170346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1048/2091] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:08 2022] PUT /identity/v3/projects/ecd5bc804a3b4f14b864fb4e0f8d1b98/users/9e3f73e469f34b66bfbe99c3d79ac129/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:08.179569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b36c09-4234-4f8a-ba54-b75b4a90ad8c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:08.180043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b36c09-4234-4f8a-ba54-b75b4a90ad8c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:08.180528 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b36c09-4234-4f8a-ba54-b75b4a90ad8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:08.210261 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-65b36c09-4234-4f8a-ba54-b75b4a90ad8c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:08.212292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-65b36c09-4234-4f8a-ba54-b75b4a90ad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:08.302015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1044/2092] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:08.313170 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de0b1a-1c3f-4167-90a9-6763d943387a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:08.316110 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d0de0b1a-1c3f-4167-90a9-6763d943387a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:08.398926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de0b1a-1c3f-4167-90a9-6763d943387a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:08.404862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287b4b7-88c5-4ba8-a526-ad088fde60df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:08.405882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287b4b7-88c5-4ba8-a526-ad088fde60df service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:08.406227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287b4b7-88c5-4ba8-a526-ad088fde60df service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:08.406600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287b4b7-88c5-4ba8-a526-ad088fde60df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:08.420012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287b4b7-88c5-4ba8-a526-ad088fde60df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:08.422286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287b4b7-88c5-4ba8-a526-ad088fde60df service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:08.435663 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1049/2093] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:05:09.350551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b36c09-4234-4f8a-ba54-b75b4a90ad8c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:09.352422 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-65b36c09-4234-4f8a-ba54-b75b4a90ad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:09.459923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b36c09-4234-4f8a-ba54-b75b4a90ad8c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:09.464882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a1d21-3811-4ed6-a4e6-05300ac7cafc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:09.466282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a1d21-3811-4ed6-a4e6-05300ac7cafc service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:09.466598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a1d21-3811-4ed6-a4e6-05300ac7cafc service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:09.466878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a1d21-3811-4ed6-a4e6-05300ac7cafc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:09.470776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3c8a1d21-3811-4ed6-a4e6-05300ac7cafc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:09.573927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:09.602839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a1d21-3811-4ed6-a4e6-05300ac7cafc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:09.606564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a1d21-3811-4ed6-a4e6-05300ac7cafc service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:09.628788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1045/2094] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:10.556524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287b4b7-88c5-4ba8-a526-ad088fde60df service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:10.558067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7287b4b7-88c5-4ba8-a526-ad088fde60df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:10.576936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287b4b7-88c5-4ba8-a526-ad088fde60df service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:10.603122 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956c863-d892-4c70-9db2-30548e60bc95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:10.604375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956c863-d892-4c70-9db2-30548e60bc95 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:10.604881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956c863-d892-4c70-9db2-30548e60bc95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:10.605497 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956c863-d892-4c70-9db2-30548e60bc95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:10.609087 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1956c863-d892-4c70-9db2-30548e60bc95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:10.754533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956c863-d892-4c70-9db2-30548e60bc95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:10.758499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956c863-d892-4c70-9db2-30548e60bc95 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:10.777499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1050/2095] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:10.991793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a1d21-3811-4ed6-a4e6-05300ac7cafc service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:11.004171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a1d21-3811-4ed6-a4e6-05300ac7cafc service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:11.012800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4430a7-cc9c-496e-8df7-307f74d37caa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:11.014943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4430a7-cc9c-496e-8df7-307f74d37caa service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:11.015612 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4430a7-cc9c-496e-8df7-307f74d37caa service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:11.016571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4430a7-cc9c-496e-8df7-307f74d37caa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:11.031082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5d4430a7-cc9c-496e-8df7-307f74d37caa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:11.216196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4430a7-cc9c-496e-8df7-307f74d37caa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:11.219039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4430a7-cc9c-496e-8df7-307f74d37caa service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:11.245481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1046/2096] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:11.983161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956c863-d892-4c70-9db2-30548e60bc95 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:12.002577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956c863-d892-4c70-9db2-30548e60bc95 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:12.020183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f0635-66bd-48e7-996e-ef4deb9e1334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:12.022206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f0635-66bd-48e7-996e-ef4deb9e1334 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:12.022883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f0635-66bd-48e7-996e-ef4deb9e1334 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:12.023539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f0635-66bd-48e7-996e-ef4deb9e1334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:12.027453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d92f0635-66bd-48e7-996e-ef4deb9e1334 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:12.092883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:12.184686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f0635-66bd-48e7-996e-ef4deb9e1334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:12.187836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f0635-66bd-48e7-996e-ef4deb9e1334 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:12.208636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1051/2097] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:14.712953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702d618-5f82-4bce-9f77-fcb105bfbe26 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:14.713550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702d618-5f82-4bce-9f77-fcb105bfbe26 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:14.713550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702d618-5f82-4bce-9f77-fcb105bfbe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:14.758104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a702d618-5f82-4bce-9f77-fcb105bfbe26 None None] MFA Rules not processed for user `9e3f73e469f34b66bfbe99c3d79ac129`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:14.761582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a702d618-5f82-4bce-9f77-fcb105bfbe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:14.923841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1047/2098] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:14.924965 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8d5ba-bb3b-4dd1-8829-bd87ad6b8b7c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:14.926057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8d5ba-bb3b-4dd1-8829-bd87ad6b8b7c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:14.926631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8d5ba-bb3b-4dd1-8829-bd87ad6b8b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:14.950560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dbd8d5ba-bb3b-4dd1-8829-bd87ad6b8b7c None None] MFA Rules not processed for user `9e3f73e469f34b66bfbe99c3d79ac129`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:14.952686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dbd8d5ba-bb3b-4dd1-8829-bd87ad6b8b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:14.965874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1052/2099] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:14.986594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702d618-5f82-4bce-9f77-fcb105bfbe26 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:14.988551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a702d618-5f82-4bce-9f77-fcb105bfbe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:15.119757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:15.130561 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702d618-5f82-4bce-9f77-fcb105bfbe26 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:15.140627 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33559c2d-05a4-4003-b303-8a50e04bb40e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:15.141857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33559c2d-05a4-4003-b303-8a50e04bb40e service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:15.142568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33559c2d-05a4-4003-b303-8a50e04bb40e service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:15.143124 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33559c2d-05a4-4003-b303-8a50e04bb40e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:15.154558 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33559c2d-05a4-4003-b303-8a50e04bb40e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:15.156644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33559c2d-05a4-4003-b303-8a50e04bb40e service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:15.179181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1048/2100] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:05:15.705809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8d5ba-bb3b-4dd1-8829-bd87ad6b8b7c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:15.707555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dbd8d5ba-bb3b-4dd1-8829-bd87ad6b8b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:15.726289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8d5ba-bb3b-4dd1-8829-bd87ad6b8b7c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:15.734509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:15.735595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:15.735881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:15.736160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] PATH_INFO: `/v3/users/3bf6e92f88954922a0507f82681396b8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:15.737918 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bf6e92f88954922a0507f82681396b8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:15.740121 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:15.884044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bf6e92f88954922a0507f82681396b8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:15.901320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bf6e92f88954922a0507f82681396b8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:15.912034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bf6e92f88954922a0507f82681396b8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:15.912714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bf6e92f88954922a0507f82681396b8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:15.929717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bf6e92f88954922a0507f82681396b8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:15.943419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1053/2101] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:15 2022] DELETE /identity/v3/users/3bf6e92f88954922a0507f82681396b8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:15.945725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0744b-7fe1-4b5b-b53d-1c1816070adb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:15.946427 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0744b-7fe1-4b5b-b53d-1c1816070adb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:15.947013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0744b-7fe1-4b5b-b53d-1c1816070adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:15.972394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2cc0744b-7fe1-4b5b-b53d-1c1816070adb None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:15.973885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2cc0744b-7fe1-4b5b-b53d-1c1816070adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:16.114526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1049/2102] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:16.129104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:16.130689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:16.250167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a4053-d652-4d37-bed5-f04d5d35d34d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:16.250895 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a4053-d652-4d37-bed5-f04d5d35d34d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:16.252590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a4053-d652-4d37-bed5-f04d5d35d34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:16.306571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b5e62-3ec2-430f-a996-a65bf8f1ee61 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:16.310572 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d7458-c0f3-4970-afb5-c63934bb3eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:16.311258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d7458-c0f3-4970-afb5-c63934bb3eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:16.311547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d7458-c0f3-4970-afb5-c63934bb3eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:16.311831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d7458-c0f3-4970-afb5-c63934bb3eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:16.316024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-694a4053-d652-4d37-bed5-f04d5d35d34d None None] MFA Rules not processed for user `9e3f73e469f34b66bfbe99c3d79ac129`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:16.318951 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-694a4053-d652-4d37-bed5-f04d5d35d34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:16.328274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d7458-c0f3-4970-afb5-c63934bb3eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:16.330701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d7458-c0f3-4970-afb5-c63934bb3eb3 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:16.358741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1054/2103] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:16.466728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1050/2104] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:16.733264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d7458-c0f3-4970-afb5-c63934bb3eb3 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:16.736301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-323d7458-c0f3-4970-afb5-c63934bb3eb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:16.777527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d7458-c0f3-4970-afb5-c63934bb3eb3 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:16.793907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:16.795466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:16.795986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:16.796474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] PATH_INFO: `/v3/users/81fa0383705d468484518c834bb8e20f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:16.798411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81fa0383705d468484518c834bb8e20f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:16.803145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:16.864359 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:16.920921 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81fa0383705d468484518c834bb8e20f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:16.986860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81fa0383705d468484518c834bb8e20f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.018609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81fa0383705d468484518c834bb8e20f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.023679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81fa0383705d468484518c834bb8e20f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.048826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81fa0383705d468484518c834bb8e20f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.069069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1055/2105] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:16 2022] DELETE /identity/v3/users/81fa0383705d468484518c834bb8e20f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:17.074482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84058d4b-e0d3-4d19-90f1-19a9c2e98a81 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:17.075485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84058d4b-e0d3-4d19-90f1-19a9c2e98a81 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:17.075960 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84058d4b-e0d3-4d19-90f1-19a9c2e98a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:17.095509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:17.109638 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:17.134524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-84058d4b-e0d3-4d19-90f1-19a9c2e98a81 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:17.138359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-84058d4b-e0d3-4d19-90f1-19a9c2e98a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:17.270246 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1051/2106] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:17.315453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84058d4b-e0d3-4d19-90f1-19a9c2e98a81 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:17.317822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-84058d4b-e0d3-4d19-90f1-19a9c2e98a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:17.332340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906700f-af2d-410c-bced-e72c4e508052 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:17.345566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:17.350991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:17.351496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:17.351805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] PATH_INFO: `/v3/projects/6cdd4e070c2843a0b86db6712855d12f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:17.356350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cdd4e070c2843a0b86db6712855d12f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:17.361161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:17.430419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:17.462330 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cdd4e070c2843a0b86db6712855d12f', 'request_id': 'req-1d46c1b9-5e3e-49bd-8998-134447f06ac0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.462995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84058d4b-e0d3-4d19-90f1-19a9c2e98a81 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:17.472107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cdd4e070c2843a0b86db6712855d12f', 'request_id': 'req-1d46c1b9-5e3e-49bd-8998-134447f06ac0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.487237 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cdd4e070c2843a0b86db6712855d12f', 'request_id': 'req-1d46c1b9-5e3e-49bd-8998-134447f06ac0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.504093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] The token cache is being invalidate because project 6cdd4e070c2843a0b86db6712855d12f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:17.504390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.505335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.507505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1056/2107] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:17 2022] DELETE /identity/v3/projects/6cdd4e070c2843a0b86db6712855d12f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:17.512450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:17.515255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:17.560850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588bd94-4399-4361-9aee-a49dbaf2a3bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:17.562220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588bd94-4399-4361-9aee-a49dbaf2a3bd service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:17.562582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588bd94-4399-4361-9aee-a49dbaf2a3bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:17.562980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588bd94-4399-4361-9aee-a49dbaf2a3bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:17.565048 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8588bd94-4399-4361-9aee-a49dbaf2a3bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:17.571644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:17.612717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46c1b9-5e3e-49bd-8998-134447f06ac0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:17.619922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:17.622193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:17.622193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:17.625542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] PATH_INFO: `/v3/users/0bc11651fe7b44e79a1bc94ae36857c5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:17.625542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bc11651fe7b44e79a1bc94ae36857c5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:17.632397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:17.721375 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588bd94-4399-4361-9aee-a49dbaf2a3bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:17.723174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588bd94-4399-4361-9aee-a49dbaf2a3bd service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:17.735716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1052/2108] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:05:17.738008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bc11651fe7b44e79a1bc94ae36857c5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.749723 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bc11651fe7b44e79a1bc94ae36857c5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.767305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bc11651fe7b44e79a1bc94ae36857c5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.768746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bc11651fe7b44e79a1bc94ae36857c5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.791148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bc11651fe7b44e79a1bc94ae36857c5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:17.801920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1057/2109] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:17 2022] DELETE /identity/v3/users/0bc11651fe7b44e79a1bc94ae36857c5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:18.409610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588bd94-4399-4361-9aee-a49dbaf2a3bd service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:18.415188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:18.416487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:18.427140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8588bd94-4399-4361-9aee-a49dbaf2a3bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:18.484181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588bd94-4399-4361-9aee-a49dbaf2a3bd service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:18.570629 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30eef55-446e-45bd-8aa0-86be998ba424 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:18.579029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:18.579953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:18.580439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:18.580829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] PATH_INFO: `/v3/projects/7662310a43b845269a2d390a8091f5f1` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:18.582900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7662310a43b845269a2d390a8091f5f1)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:18.585754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:18.622112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:18.623972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:18.624416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:18.624769 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] PATH_INFO: `/v3/projects/ed4cebafbf2144d0b8e03b1756d502cf` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:18.626695 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed4cebafbf2144d0b8e03b1756d502cf)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:18.631037 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:18.688882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7662310a43b845269a2d390a8091f5f1', 'request_id': 'req-9716bdb3-5d42-4750-82da-ae7547c523b3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:18.693300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:18.717325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7662310a43b845269a2d390a8091f5f1', 'request_id': 'req-9716bdb3-5d42-4750-82da-ae7547c523b3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:18.735574 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:18.748785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7662310a43b845269a2d390a8091f5f1', 'request_id': 'req-9716bdb3-5d42-4750-82da-ae7547c523b3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:18.754640 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed4cebafbf2144d0b8e03b1756d502cf', 'request_id': 'req-a057a351-edc3-43f0-947e-cc968c1edf7a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:18.764462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed4cebafbf2144d0b8e03b1756d502cf', 'request_id': 'req-a057a351-edc3-43f0-947e-cc968c1edf7a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:18.775860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed4cebafbf2144d0b8e03b1756d502cf', 'request_id': 'req-a057a351-edc3-43f0-947e-cc968c1edf7a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:18.790035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] The token cache is being invalidate because project 7662310a43b845269a2d390a8091f5f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:18.790368 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:18.791445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:18.795431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1058/2110] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:18 2022] DELETE /identity/v3/projects/7662310a43b845269a2d390a8091f5f1 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:18.802467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:18.805092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:18.810969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] The token cache is being invalidate because project ed4cebafbf2144d0b8e03b1756d502cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:18.811905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:18.813168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:18.822922 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1053/2111] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:18 2022] DELETE /identity/v3/projects/ed4cebafbf2144d0b8e03b1756d502cf => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:18.828677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:18.834834 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:19.014414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716bdb3-5d42-4750-82da-ae7547c523b3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:19.027974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057a351-edc3-43f0-947e-cc968c1edf7a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:19.033667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:19.035138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:19.035656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:19.036145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] PATH_INFO: `/v3/users/2db5d8a156994821a4fb0a14f83a2665` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:19.043884 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2db5d8a156994821a4fb0a14f83a2665)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:19.047100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:19.052267 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314b895-0f79-49c3-969d-0d94cf43cc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:19.054082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314b895-0f79-49c3-969d-0d94cf43cc6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:19.054364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314b895-0f79-49c3-969d-0d94cf43cc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:19.054643 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314b895-0f79-49c3-969d-0d94cf43cc6d admin admin] PATH_INFO: `/v3/users/02c00e1e3c9440ff9b0601a58e9096cb` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:19.056486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314b895-0f79-49c3-969d-0d94cf43cc6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=02c00e1e3c9440ff9b0601a58e9096cb)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:19.060268 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314b895-0f79-49c3-969d-0d94cf43cc6d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:19.146119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f314b895-0f79-49c3-969d-0d94cf43cc6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02c00e1e3c9440ff9b0601a58e9096cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:19.148126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f314b895-0f79-49c3-969d-0d94cf43cc6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02c00e1e3c9440ff9b0601a58e9096cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:19.149229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f314b895-0f79-49c3-969d-0d94cf43cc6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02c00e1e3c9440ff9b0601a58e9096cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:19.152350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2db5d8a156994821a4fb0a14f83a2665'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:19.163167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2db5d8a156994821a4fb0a14f83a2665'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:19.165107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f314b895-0f79-49c3-969d-0d94cf43cc6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02c00e1e3c9440ff9b0601a58e9096cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:19.173488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2db5d8a156994821a4fb0a14f83a2665'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:19.174702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2db5d8a156994821a4fb0a14f83a2665'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:19.188054 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f314b895-0f79-49c3-969d-0d94cf43cc6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02c00e1e3c9440ff9b0601a58e9096cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:19.192366 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2db5d8a156994821a4fb0a14f83a2665'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:19.209163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1059/2112] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:18 2022] DELETE /identity/v3/users/2db5d8a156994821a4fb0a14f83a2665 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:19.246877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1054/2113] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:18 2022] DELETE /identity/v3/users/02c00e1e3c9440ff9b0601a58e9096cb => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:19.775215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:19.777757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:19.948930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc99e0-6fd1-47d0-bd8a-627a14fd141a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:19.957218 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:20.101032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:20.103078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:20.103718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:20.104472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] PATH_INFO: `/v3/projects/43cfbe79e8fa4903994fb79e045a3bae` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:20.107329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43cfbe79e8fa4903994fb79e045a3bae)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:20.113293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:20.339083 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43cfbe79e8fa4903994fb79e045a3bae', 'request_id': 'req-2837b691-9709-47c6-a270-e2d40eded540'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:20.360829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43cfbe79e8fa4903994fb79e045a3bae', 'request_id': 'req-2837b691-9709-47c6-a270-e2d40eded540'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:20.376871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43cfbe79e8fa4903994fb79e045a3bae', 'request_id': 'req-2837b691-9709-47c6-a270-e2d40eded540'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:20.412744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] The token cache is being invalidate because project 43cfbe79e8fa4903994fb79e045a3bae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:20.413358 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:20.419339 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:20.422617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1060/2114] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:19 2022] DELETE /identity/v3/projects/43cfbe79e8fa4903994fb79e045a3bae => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:20.429592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7fbe5-5be3-492e-9d18-839c3ba79c52 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:20.430189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7fbe5-5be3-492e-9d18-839c3ba79c52 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:20.430290 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7fbe5-5be3-492e-9d18-839c3ba79c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:20.463396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d5d7fbe5-5be3-492e-9d18-839c3ba79c52 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:20.466628 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d5d7fbe5-5be3-492e-9d18-839c3ba79c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:20.563109 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:20.566300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:20.650075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:20.695078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1055/2115] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:20.697010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a7098-c0fe-48b3-9b17-65c01d0746d8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:20.697409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a7098-c0fe-48b3-9b17-65c01d0746d8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:20.700878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a7098-c0fe-48b3-9b17-65c01d0746d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:20.751752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f70a7098-c0fe-48b3-9b17-65c01d0746d8 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:20.753497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f70a7098-c0fe-48b3-9b17-65c01d0746d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:20.782119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1056/2116] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:20.787960 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a7098-c0fe-48b3-9b17-65c01d0746d8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:20.808286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a7098-c0fe-48b3-9b17-65c01d0746d8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:20.821970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16167b-1980-4cc0-8913-b9f89f9e007c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:20.822780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16167b-1980-4cc0-8913-b9f89f9e007c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:20.823313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16167b-1980-4cc0-8913-b9f89f9e007c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:20.823603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16167b-1980-4cc0-8913-b9f89f9e007c admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:20.824502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16167b-1980-4cc0-8913-b9f89f9e007c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:05:20.824891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16167b-1980-4cc0-8913-b9f89f9e007c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:20.826610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16167b-1980-4cc0-8913-b9f89f9e007c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:20.873974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837b691-9709-47c6-a270-e2d40eded540 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:20.875780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1057/2117] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:05:20.879869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16167b-1980-4cc0-8913-b9f89f9e007c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:20.886740 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:20.887892 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:20.890987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:20.891402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] PATH_INFO: `/v3/projects/af82ac7930ee49e8b26b0137aa1619a9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:20.894833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] RBAC: Authorizing `identity:delete_project(project_id=af82ac7930ee49e8b26b0137aa1619a9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:20.898134 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:20.906899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16167b-1980-4cc0-8913-b9f89f9e007c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:20.921785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a4e93-8bd4-47a8-a8e9-28422ebf3f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:20.922656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a4e93-8bd4-47a8-a8e9-28422ebf3f22 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:20.923034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a4e93-8bd4-47a8-a8e9-28422ebf3f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:20.923407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a4e93-8bd4-47a8-a8e9-28422ebf3f22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:20.924353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a4e93-8bd4-47a8-a8e9-28422ebf3f22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:20.929937 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a4e93-8bd4-47a8-a8e9-28422ebf3f22 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:20.979800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:20.990015 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af82ac7930ee49e8b26b0137aa1619a9', 'request_id': 'req-45fd1d94-2097-4fda-809c-ad7df837617e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:21.000419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af82ac7930ee49e8b26b0137aa1619a9', 'request_id': 'req-45fd1d94-2097-4fda-809c-ad7df837617e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:21.015791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af82ac7930ee49e8b26b0137aa1619a9', 'request_id': 'req-45fd1d94-2097-4fda-809c-ad7df837617e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:21.021708 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1058/2118] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:05:20 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:21.026149 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a4e93-8bd4-47a8-a8e9-28422ebf3f22 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:21.055687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] The token cache is being invalidate because project af82ac7930ee49e8b26b0137aa1619a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:21.056032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:21.058011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:21.062511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1061/2119] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:20 2022] DELETE /identity/v3/projects/af82ac7930ee49e8b26b0137aa1619a9 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:05:21.068863 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:21.076450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:21.147643 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a4e93-8bd4-47a8-a8e9-28422ebf3f22 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:21.168820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f53a5f-2298-4334-8c69-df283e3352ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:21.173445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f53a5f-2298-4334-8c69-df283e3352ce admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:21.174346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f53a5f-2298-4334-8c69-df283e3352ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:21.174917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f53a5f-2298-4334-8c69-df283e3352ce admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:21.175869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f53a5f-2298-4334-8c69-df283e3352ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:21.184472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f53a5f-2298-4334-8c69-df283e3352ce admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:21.259986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd1d94-2097-4fda-809c-ad7df837617e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:21.264316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1059/2120] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:05:21 2022] POST /identity/v3/users => generated 291 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:21.268348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f53a5f-2298-4334-8c69-df283e3352ce admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:21.271195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-73f53a5f-2298-4334-8c69-df283e3352ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:21.271623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c928ed8-9e13-4f9b-be0e-334d74adc354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:21.272575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c928ed8-9e13-4f9b-be0e-334d74adc354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:21.272860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c928ed8-9e13-4f9b-be0e-334d74adc354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:21.273139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c928ed8-9e13-4f9b-be0e-334d74adc354 admin admin] PATH_INFO: `/v3/users/d194c9cb0bc946c0a057b2c4653d53fe` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:21.279108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c928ed8-9e13-4f9b-be0e-334d74adc354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d194c9cb0bc946c0a057b2c4653d53fe)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:21.281573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c928ed8-9e13-4f9b-be0e-334d74adc354 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:21.304622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f53a5f-2298-4334-8c69-df283e3352ce admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:21.314235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65026a3c-f8a1-4ac3-8a96-386e36863b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:21.315851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65026a3c-f8a1-4ac3-8a96-386e36863b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:21.316377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65026a3c-f8a1-4ac3-8a96-386e36863b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:21.327794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65026a3c-f8a1-4ac3-8a96-386e36863b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:21.328619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65026a3c-f8a1-4ac3-8a96-386e36863b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:21.330632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65026a3c-f8a1-4ac3-8a96-386e36863b57 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:21.347219 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1060/2121] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:21.351412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65026a3c-f8a1-4ac3-8a96-386e36863b57 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:21.363651 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65026a3c-f8a1-4ac3-8a96-386e36863b57 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:21.369091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9a7d8-4da5-4e4f-9ff7-96015d65c47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:21.369965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9a7d8-4da5-4e4f-9ff7-96015d65c47d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:21.370256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9a7d8-4da5-4e4f-9ff7-96015d65c47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:21.370587 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9a7d8-4da5-4e4f-9ff7-96015d65c47d admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:21.371326 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9a7d8-4da5-4e4f-9ff7-96015d65c47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:21.372998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9a7d8-4da5-4e4f-9ff7-96015d65c47d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:21.382665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6c928ed8-9e13-4f9b-be0e-334d74adc354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd194c9cb0bc946c0a057b2c4653d53fe'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:21.388499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1061/2122] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:05:21.393677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9a7d8-4da5-4e4f-9ff7-96015d65c47d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:21.400167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6c928ed8-9e13-4f9b-be0e-334d74adc354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd194c9cb0bc946c0a057b2c4653d53fe'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:21.413134 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6c928ed8-9e13-4f9b-be0e-334d74adc354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd194c9cb0bc946c0a057b2c4653d53fe'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:21.414313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6c928ed8-9e13-4f9b-be0e-334d74adc354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd194c9cb0bc946c0a057b2c4653d53fe'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:21.428521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9a7d8-4da5-4e4f-9ff7-96015d65c47d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:21.451015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2280612-1c6f-4291-a60a-60b16d59f8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:21.452049 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2280612-1c6f-4291-a60a-60b16d59f8c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:21.452337 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2280612-1c6f-4291-a60a-60b16d59f8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:21.452618 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2280612-1c6f-4291-a60a-60b16d59f8c4 admin admin] PATH_INFO: `/v3/projects/6de1b2d9c55f4af7b6793ac472a6d408/users/34b2b4070d344d7ca07a99985678467a/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:21.459746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6c928ed8-9e13-4f9b-be0e-334d74adc354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd194c9cb0bc946c0a057b2c4653d53fe'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:21.480023 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1062/2123] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:21 2022] DELETE /identity/v3/users/d194c9cb0bc946c0a057b2c4653d53fe => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:21.505396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2280612-1c6f-4291-a60a-60b16d59f8c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de1b2d9c55f4af7b6793ac472a6d408, user_id=34b2b4070d344d7ca07a99985678467a, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:21.508290 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2280612-1c6f-4291-a60a-60b16d59f8c4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:21.537977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1062/2124] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:21 2022] PUT /identity/v3/projects/6de1b2d9c55f4af7b6793ac472a6d408/users/34b2b4070d344d7ca07a99985678467a/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:21.552326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54b5d9-b118-4f83-8c60-2851a233d226 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:21.552764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54b5d9-b118-4f83-8c60-2851a233d226 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:21.553130 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54b5d9-b118-4f83-8c60-2851a233d226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:21.594848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cc54b5d9-b118-4f83-8c60-2851a233d226 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:21.596585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cc54b5d9-b118-4f83-8c60-2851a233d226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:21.672235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:21.716926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1063/2125] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:21.757665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2280612-1c6f-4291-a60a-60b16d59f8c4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:21.760090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e2280612-1c6f-4291-a60a-60b16d59f8c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:21.917613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2280612-1c6f-4291-a60a-60b16d59f8c4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:21.923176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f0c47-4069-4dee-8dff-693ae3c76c8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:21.924620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0c47-4069-4dee-8dff-693ae3c76c8f service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:21.925848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0c47-4069-4dee-8dff-693ae3c76c8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:21.926306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0c47-4069-4dee-8dff-693ae3c76c8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:21.947340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:21.951234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f0c47-4069-4dee-8dff-693ae3c76c8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:21.953991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f0c47-4069-4dee-8dff-693ae3c76c8f service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:21.969497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1063/2126] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:05:22.038978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54b5d9-b118-4f83-8c60-2851a233d226 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:22.042367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cc54b5d9-b118-4f83-8c60-2851a233d226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:22.072006 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54b5d9-b118-4f83-8c60-2851a233d226 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:22.085905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:22.086819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:22.087134 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:22.087675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f admin admin] PATH_INFO: `/v3/projects/340e27bbd3f143ad908b3543cc25bc8a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:22.090448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=340e27bbd3f143ad908b3543cc25bc8a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:22.092431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:22.222781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340e27bbd3f143ad908b3543cc25bc8a', 'request_id': 'req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:22.231940 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340e27bbd3f143ad908b3543cc25bc8a', 'request_id': 'req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:22.243181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340e27bbd3f143ad908b3543cc25bc8a', 'request_id': 'req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:22.265735 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f admin admin] The token cache is being invalidate because project 340e27bbd3f143ad908b3543cc25bc8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:22.266036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:22.266755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fc88eca7-4dd7-4848-b886-92bb4dba4c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:22.272251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1064/2127] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:22 2022] DELETE /identity/v3/projects/340e27bbd3f143ad908b3543cc25bc8a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:22.274582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d595d-bb30-43d5-8aad-5af6d49167b6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:22.274974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d595d-bb30-43d5-8aad-5af6d49167b6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:22.275351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d595d-bb30-43d5-8aad-5af6d49167b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:22.305242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-664d595d-bb30-43d5-8aad-5af6d49167b6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:22.306558 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-664d595d-bb30-43d5-8aad-5af6d49167b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:22.406318 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1064/2128] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:05:22.484883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ee997-0919-48ce-b9fa-6455e51cb6ef None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:22.485394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ee997-0919-48ce-b9fa-6455e51cb6ef None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:22.485565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ee997-0919-48ce-b9fa-6455e51cb6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:22.533196 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-550ee997-0919-48ce-b9fa-6455e51cb6ef None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:22.537274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-550ee997-0919-48ce-b9fa-6455e51cb6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:22.550758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1065/2129] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:22.558152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d595d-bb30-43d5-8aad-5af6d49167b6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:22.572468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d595d-bb30-43d5-8aad-5af6d49167b6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:22.579339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc14c9-238a-4f24-89cf-32a021147da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:22.580646 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc14c9-238a-4f24-89cf-32a021147da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:22.581190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc14c9-238a-4f24-89cf-32a021147da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:22.581531 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc14c9-238a-4f24-89cf-32a021147da6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:22.582384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc14c9-238a-4f24-89cf-32a021147da6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:05:22.583024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc14c9-238a-4f24-89cf-32a021147da6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:22.584914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc14c9-238a-4f24-89cf-32a021147da6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:22.603344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1065/2130] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:05:22.610359 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ee997-0919-48ce-b9fa-6455e51cb6ef None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:22.630000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ee997-0919-48ce-b9fa-6455e51cb6ef None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:22.653273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1eb5b2-a156-4914-90a9-ba0e8d8d0498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:22.654660 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1eb5b2-a156-4914-90a9-ba0e8d8d0498 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:22.655027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1eb5b2-a156-4914-90a9-ba0e8d8d0498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:22.655559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1eb5b2-a156-4914-90a9-ba0e8d8d0498 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:22.656910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1eb5b2-a156-4914-90a9-ba0e8d8d0498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:22.661867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1eb5b2-a156-4914-90a9-ba0e8d8d0498 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:22.702312 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1066/2131] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:05:22 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:22.704157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc14c9-238a-4f24-89cf-32a021147da6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:22.717796 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc14c9-238a-4f24-89cf-32a021147da6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:22.812804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec37dab-8d5d-4928-a282-d1f06644b6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:22.813537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec37dab-8d5d-4928-a282-d1f06644b6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:22.814052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec37dab-8d5d-4928-a282-d1f06644b6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:22.814540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec37dab-8d5d-4928-a282-d1f06644b6b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:22.815596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec37dab-8d5d-4928-a282-d1f06644b6b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:22.817677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec37dab-8d5d-4928-a282-d1f06644b6b6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:22.869409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1066/2132] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:05:22 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:22.871178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1eb5b2-a156-4914-90a9-ba0e8d8d0498 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:22.906797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1eb5b2-a156-4914-90a9-ba0e8d8d0498 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:22.929148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b572681-a780-4a9a-a73a-9b54ba86b884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:22.930333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b572681-a780-4a9a-a73a-9b54ba86b884 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:22.930829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b572681-a780-4a9a-a73a-9b54ba86b884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:22.931298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b572681-a780-4a9a-a73a-9b54ba86b884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:22.932864 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b572681-a780-4a9a-a73a-9b54ba86b884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:22.936360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b572681-a780-4a9a-a73a-9b54ba86b884 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:22.972160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1067/2133] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:22.977354 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec37dab-8d5d-4928-a282-d1f06644b6b6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:22.998935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec37dab-8d5d-4928-a282-d1f06644b6b6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:23.029777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cea59-d3da-483a-a2fb-0bfdba5c685d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:23.030486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cea59-d3da-483a-a2fb-0bfdba5c685d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:23.030777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cea59-d3da-483a-a2fb-0bfdba5c685d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:23.031055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cea59-d3da-483a-a2fb-0bfdba5c685d admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:23.031819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cea59-d3da-483a-a2fb-0bfdba5c685d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:23.035108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cea59-d3da-483a-a2fb-0bfdba5c685d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:23.055693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1067/2134] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:23.060561 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b572681-a780-4a9a-a73a-9b54ba86b884 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:23.082187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b572681-a780-4a9a-a73a-9b54ba86b884 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:23.100197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04557e5c-dc27-4804-b5c7-5da5133d2a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:23.101354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04557e5c-dc27-4804-b5c7-5da5133d2a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:23.101770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04557e5c-dc27-4804-b5c7-5da5133d2a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:23.102091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04557e5c-dc27-4804-b5c7-5da5133d2a54 admin admin] PATH_INFO: `/v3/projects/753d5874ae814841af6314a4db696221/users/1fd15df599fa4261b917f4f9b1d1f75a/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:23.129146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04557e5c-dc27-4804-b5c7-5da5133d2a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=753d5874ae814841af6314a4db696221, user_id=1fd15df599fa4261b917f4f9b1d1f75a, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:23.132075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04557e5c-dc27-4804-b5c7-5da5133d2a54 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:23.156333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1068/2135] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:23 2022] PUT /identity/v3/projects/753d5874ae814841af6314a4db696221/users/1fd15df599fa4261b917f4f9b1d1f75a/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:23.175087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41706d-d662-47c7-a6c0-ffe5dd949523 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:23.175697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41706d-d662-47c7-a6c0-ffe5dd949523 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:23.176093 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41706d-d662-47c7-a6c0-ffe5dd949523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:23.211038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ef41706d-d662-47c7-a6c0-ffe5dd949523 None None] MFA Rules not processed for user `1fd15df599fa4261b917f4f9b1d1f75a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:23.213314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ef41706d-d662-47c7-a6c0-ffe5dd949523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:23.297114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:23.327183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1068/2136] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:23.344207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04557e5c-dc27-4804-b5c7-5da5133d2a54 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:23.346363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-04557e5c-dc27-4804-b5c7-5da5133d2a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:23.490779 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04557e5c-dc27-4804-b5c7-5da5133d2a54 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:23.496298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438efe2-6df1-4cd2-9664-db09edc4a94c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:23.497034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438efe2-6df1-4cd2-9664-db09edc4a94c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:23.497363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438efe2-6df1-4cd2-9664-db09edc4a94c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:23.497655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438efe2-6df1-4cd2-9664-db09edc4a94c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:23.508719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438efe2-6df1-4cd2-9664-db09edc4a94c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:23.510236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438efe2-6df1-4cd2-9664-db09edc4a94c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:23.533425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1069/2137] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:23.785727 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a179398-6ed9-4bea-94c2-1aeee69b4a59 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:23.786282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a179398-6ed9-4bea-94c2-1aeee69b4a59 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:23.786383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a179398-6ed9-4bea-94c2-1aeee69b4a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:23.821820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5a179398-6ed9-4bea-94c2-1aeee69b4a59 None None] MFA Rules not processed for user `34b2b4070d344d7ca07a99985678467a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:23.823541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5a179398-6ed9-4bea-94c2-1aeee69b4a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:23.936887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1069/2138] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:23.977360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438efe2-6df1-4cd2-9664-db09edc4a94c service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:23.978929 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f438efe2-6df1-4cd2-9664-db09edc4a94c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:23.995227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438efe2-6df1-4cd2-9664-db09edc4a94c service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:24.000181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4dee6-a75a-426c-bb7b-84fe01d53265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:24.000181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4dee6-a75a-426c-bb7b-84fe01d53265 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:24.000473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4dee6-a75a-426c-bb7b-84fe01d53265 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:24.000756 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4dee6-a75a-426c-bb7b-84fe01d53265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:24.009413 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:24.016818 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4dee6-a75a-426c-bb7b-84fe01d53265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:24.018941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4dee6-a75a-426c-bb7b-84fe01d53265 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:24.057443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1070/2139] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:05:25.131520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c598c1-47fa-4948-bc1d-76b80fd65378 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:25.133271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c598c1-47fa-4948-bc1d-76b80fd65378 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:25.133835 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c598c1-47fa-4948-bc1d-76b80fd65378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:25.157834 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f9c598c1-47fa-4948-bc1d-76b80fd65378 None None] MFA Rules not processed for user `1fd15df599fa4261b917f4f9b1d1f75a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:25.159606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f9c598c1-47fa-4948-bc1d-76b80fd65378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:25.180658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1070/2140] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:25.573273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c046dc-b443-45fc-bed0-a774878eafb6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:25.573964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c046dc-b443-45fc-bed0-a774878eafb6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:25.573964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c046dc-b443-45fc-bed0-a774878eafb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:25.598848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b1c046dc-b443-45fc-bed0-a774878eafb6 None None] MFA Rules not processed for user `34b2b4070d344d7ca07a99985678467a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:25.600909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b1c046dc-b443-45fc-bed0-a774878eafb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:25.620169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1071/2141] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:35.150876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=39 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:35.590393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:39.323425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c598c1-47fa-4948-bc1d-76b80fd65378 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:39.325978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f9c598c1-47fa-4948-bc1d-76b80fd65378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:39.417746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c598c1-47fa-4948-bc1d-76b80fd65378 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:39.423325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:39.425097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:39.425611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:39.426239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] PATH_INFO: `/v3/users/e67aa374e7874bd6b484c6180c1a5deb` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:39.428201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e67aa374e7874bd6b484c6180c1a5deb)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:39.430622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:39.503642 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e67aa374e7874bd6b484c6180c1a5deb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:39.504689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e67aa374e7874bd6b484c6180c1a5deb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:39.505344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e67aa374e7874bd6b484c6180c1a5deb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:39.515168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e67aa374e7874bd6b484c6180c1a5deb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:39.550219 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e67aa374e7874bd6b484c6180c1a5deb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:39.592226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1071/2142] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:39 2022] DELETE /identity/v3/users/e67aa374e7874bd6b484c6180c1a5deb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:39.597221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbff2f3-4491-4de7-b374-78d8c00ec615 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:39.597844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbff2f3-4491-4de7-b374-78d8c00ec615 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:39.597844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbff2f3-4491-4de7-b374-78d8c00ec615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:39.622127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ecbff2f3-4491-4de7-b374-78d8c00ec615 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:39.624483 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ecbff2f3-4491-4de7-b374-78d8c00ec615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:39.723965 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1072/2143] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:05:39.735272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:39.737158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:39.863344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f492126-80b2-4d93-93a9-a02a6d01a166 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:39.870892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc186fe1-a10f-4653-a0cf-304e1a7b699c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:39.872145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc186fe1-a10f-4653-a0cf-304e1a7b699c service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:39.872602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc186fe1-a10f-4653-a0cf-304e1a7b699c service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:39.872997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc186fe1-a10f-4653-a0cf-304e1a7b699c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:39.891052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc186fe1-a10f-4653-a0cf-304e1a7b699c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:39.894437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc186fe1-a10f-4653-a0cf-304e1a7b699c service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:39.913448 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1072/2144] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:40.617884 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbff2f3-4491-4de7-b374-78d8c00ec615 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:40.619776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ecbff2f3-4491-4de7-b374-78d8c00ec615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:40.646734 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbff2f3-4491-4de7-b374-78d8c00ec615 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:40.659446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:40.660429 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:40.660913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:40.661530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc admin admin] PATH_INFO: `/v3/projects/f2416194316d4088b02f35a6673dd007` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:40.663461 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2416194316d4088b02f35a6673dd007)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:40.665876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:40.761450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2416194316d4088b02f35a6673dd007', 'request_id': 'req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:40.776213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2416194316d4088b02f35a6673dd007', 'request_id': 'req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:40.794327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2416194316d4088b02f35a6673dd007', 'request_id': 'req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:40.821669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc admin admin] The token cache is being invalidate because project f2416194316d4088b02f35a6673dd007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:40.822472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:40.824254 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-766efd7e-7085-4bd0-a801-5c3bcdf8d5bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:40.828335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1073/2145] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:40 2022] DELETE /identity/v3/projects/f2416194316d4088b02f35a6673dd007 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:40.835750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc466487-7201-4397-83eb-2f8f14032980 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:40.836154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc466487-7201-4397-83eb-2f8f14032980 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:40.836650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc466487-7201-4397-83eb-2f8f14032980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:40.860537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cc466487-7201-4397-83eb-2f8f14032980 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:40.862716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cc466487-7201-4397-83eb-2f8f14032980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:40.886843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:40.951312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1073/2146] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:40.964907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f87d9-7e93-4d5f-8f8a-3e4923c09052 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:40.966042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f87d9-7e93-4d5f-8f8a-3e4923c09052 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:40.967302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f87d9-7e93-4d5f-8f8a-3e4923c09052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:41.009168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a73f87d9-7e93-4d5f-8f8a-3e4923c09052 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:41.010526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a73f87d9-7e93-4d5f-8f8a-3e4923c09052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:41.020580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1074/2147] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:41.027302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc466487-7201-4397-83eb-2f8f14032980 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:41.045970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc466487-7201-4397-83eb-2f8f14032980 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:41.056669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb1f69-be9d-4351-b75c-915d7a9ddf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:41.057909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb1f69-be9d-4351-b75c-915d7a9ddf45 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:41.058651 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb1f69-be9d-4351-b75c-915d7a9ddf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:41.059174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb1f69-be9d-4351-b75c-915d7a9ddf45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:41.060250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb1f69-be9d-4351-b75c-915d7a9ddf45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:05:41.060912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb1f69-be9d-4351-b75c-915d7a9ddf45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:41.065076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb1f69-be9d-4351-b75c-915d7a9ddf45 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:41.088183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1074/2148] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:05:41.094822 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f87d9-7e93-4d5f-8f8a-3e4923c09052 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:41.114529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f87d9-7e93-4d5f-8f8a-3e4923c09052 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:41.126177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85304d65-c5cc-43ca-b2a4-39e04da61d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:41.128479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85304d65-c5cc-43ca-b2a4-39e04da61d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:41.128968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85304d65-c5cc-43ca-b2a4-39e04da61d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:41.129260 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85304d65-c5cc-43ca-b2a4-39e04da61d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:41.130355 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85304d65-c5cc-43ca-b2a4-39e04da61d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:41.133030 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85304d65-c5cc-43ca-b2a4-39e04da61d0c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:41.169678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1075/2149] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:05:41 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:41.173083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb1f69-be9d-4351-b75c-915d7a9ddf45 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:41.186444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb1f69-be9d-4351-b75c-915d7a9ddf45 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:41.337563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dab434-aafe-479f-9101-21767e7546d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:41.338681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dab434-aafe-479f-9101-21767e7546d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:41.338987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dab434-aafe-479f-9101-21767e7546d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:41.339424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dab434-aafe-479f-9101-21767e7546d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:41.339976 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dab434-aafe-479f-9101-21767e7546d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:41.342049 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dab434-aafe-479f-9101-21767e7546d2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:41.371859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1075/2150] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:05:41 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:05:41.378292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85304d65-c5cc-43ca-b2a4-39e04da61d0c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:41.390851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85304d65-c5cc-43ca-b2a4-39e04da61d0c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:41.398943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1c4bf-6352-49ed-9402-bc7a226391f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:41.400084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1c4bf-6352-49ed-9402-bc7a226391f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:41.400644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1c4bf-6352-49ed-9402-bc7a226391f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:41.402885 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1c4bf-6352-49ed-9402-bc7a226391f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:41.404613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1c4bf-6352-49ed-9402-bc7a226391f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:41.407908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1c4bf-6352-49ed-9402-bc7a226391f4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:41.429392 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1076/2151] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:41.434052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dab434-aafe-479f-9101-21767e7546d2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:41.452320 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dab434-aafe-479f-9101-21767e7546d2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:41.462715 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d29f9-8949-470a-a5ef-3532650be87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:41.463989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d29f9-8949-470a-a5ef-3532650be87a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:41.464504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d29f9-8949-470a-a5ef-3532650be87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:41.465021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d29f9-8949-470a-a5ef-3532650be87a admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:41.466474 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d29f9-8949-470a-a5ef-3532650be87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:41.470141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d29f9-8949-470a-a5ef-3532650be87a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:41.497769 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1076/2152] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:41.500719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1c4bf-6352-49ed-9402-bc7a226391f4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:41.524116 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1c4bf-6352-49ed-9402-bc7a226391f4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:41.533999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef743fc-7283-4934-9e70-a2d8f601b367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:41.536124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef743fc-7283-4934-9e70-a2d8f601b367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:41.536991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef743fc-7283-4934-9e70-a2d8f601b367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:41.537811 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef743fc-7283-4934-9e70-a2d8f601b367 admin admin] PATH_INFO: `/v3/projects/391276cbafe741279b381766f32c8bad/users/3c4fcc67f0e240af8f1437e9b9b757aa/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:41.578532 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef743fc-7283-4934-9e70-a2d8f601b367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=391276cbafe741279b381766f32c8bad, user_id=3c4fcc67f0e240af8f1437e9b9b757aa, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:41.585332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef743fc-7283-4934-9e70-a2d8f601b367 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:41.620435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1077/2153] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:41 2022] PUT /identity/v3/projects/391276cbafe741279b381766f32c8bad/users/3c4fcc67f0e240af8f1437e9b9b757aa/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:41.641577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647b393-b22b-4bdf-99c2-e82e93e49ecf None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:41.642530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647b393-b22b-4bdf-99c2-e82e93e49ecf None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:41.643290 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647b393-b22b-4bdf-99c2-e82e93e49ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:41.725590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3647b393-b22b-4bdf-99c2-e82e93e49ecf None None] MFA Rules not processed for user `3c4fcc67f0e240af8f1437e9b9b757aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:41.727734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3647b393-b22b-4bdf-99c2-e82e93e49ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:41.820027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0ed1b-e873-48e3-b87a-29cdb8c312a7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:41.821499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0ed1b-e873-48e3-b87a-29cdb8c312a7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:41.822225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0ed1b-e873-48e3-b87a-29cdb8c312a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:41.855285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8af0ed1b-e873-48e3-b87a-29cdb8c312a7 None None] MFA Rules not processed for user `126ba607541e4c4895b47b4b7602ce01`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:41.857784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8af0ed1b-e873-48e3-b87a-29cdb8c312a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:41.877873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1077/2154] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:41.889825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:41.894601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647b393-b22b-4bdf-99c2-e82e93e49ecf None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:41.897022 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3647b393-b22b-4bdf-99c2-e82e93e49ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:41.926382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:41.965188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1078/2155] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:42.015883 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647b393-b22b-4bdf-99c2-e82e93e49ecf None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:42.021156 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916a232-28d8-453c-8854-42d9c22afab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:42.021875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916a232-28d8-453c-8854-42d9c22afab0 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:42.022243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916a232-28d8-453c-8854-42d9c22afab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:42.022413 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916a232-28d8-453c-8854-42d9c22afab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:42.032470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916a232-28d8-453c-8854-42d9c22afab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:42.035053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916a232-28d8-453c-8854-42d9c22afab0 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:42.046155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1078/2156] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:42.084384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0ed1b-e873-48e3-b87a-29cdb8c312a7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:42.216841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0ed1b-e873-48e3-b87a-29cdb8c312a7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:42.227486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:42.228316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:42.228821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:42.229297 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] PATH_INFO: `/v3/users/3c4fcc67f0e240af8f1437e9b9b757aa` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:42.231845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c4fcc67f0e240af8f1437e9b9b757aa)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:42.237751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:42.351332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c4fcc67f0e240af8f1437e9b9b757aa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:42.364311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c4fcc67f0e240af8f1437e9b9b757aa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:42.377791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c4fcc67f0e240af8f1437e9b9b757aa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:42.378964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c4fcc67f0e240af8f1437e9b9b757aa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:42.407313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c4fcc67f0e240af8f1437e9b9b757aa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:42.424128 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1079/2157] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:42 2022] DELETE /identity/v3/users/3c4fcc67f0e240af8f1437e9b9b757aa => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:42.429313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1720bf2-c43b-402b-9d47-7ca02b3de5da None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:42.429594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1720bf2-c43b-402b-9d47-7ca02b3de5da None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:42.429914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1720bf2-c43b-402b-9d47-7ca02b3de5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:42.458564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b1720bf2-c43b-402b-9d47-7ca02b3de5da None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:42.460562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b1720bf2-c43b-402b-9d47-7ca02b3de5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:42.558466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1079/2158] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:05:42.575256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:42.577817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:42.690824 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:42.717653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35f647-2725-4637-b0e1-fcd49739f5a1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:42.728001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4029c6-dee1-4ae8-84a6-ddde609b5715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:42.729151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4029c6-dee1-4ae8-84a6-ddde609b5715 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:42.729837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4029c6-dee1-4ae8-84a6-ddde609b5715 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:42.731384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4029c6-dee1-4ae8-84a6-ddde609b5715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:42.755517 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4029c6-dee1-4ae8-84a6-ddde609b5715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:42.759290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4029c6-dee1-4ae8-84a6-ddde609b5715 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:42.791193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1080/2159] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:05:43.461568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1720bf2-c43b-402b-9d47-7ca02b3de5da None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:43.463141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b1720bf2-c43b-402b-9d47-7ca02b3de5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:43.480529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1720bf2-c43b-402b-9d47-7ca02b3de5da None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:43.486261 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:43.487479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:43.487763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:43.488040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb admin admin] PATH_INFO: `/v3/projects/391276cbafe741279b381766f32c8bad` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:43.489728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=391276cbafe741279b381766f32c8bad)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:43.491954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:43.645214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '391276cbafe741279b381766f32c8bad', 'request_id': 'req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:43.657141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '391276cbafe741279b381766f32c8bad', 'request_id': 'req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:43.667502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '391276cbafe741279b381766f32c8bad', 'request_id': 'req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:43.691852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb admin admin] The token cache is being invalidate because project 391276cbafe741279b381766f32c8bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:43.692532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:43.693560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-198b4b8e-42fc-4d00-93e4-78c1b59a35eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:43.695926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1080/2160] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:43 2022] DELETE /identity/v3/projects/391276cbafe741279b381766f32c8bad => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:43.703411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de080ce2-c0f1-47cb-8245-a92ecce17c1d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:43.704284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de080ce2-c0f1-47cb-8245-a92ecce17c1d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:43.704867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de080ce2-c0f1-47cb-8245-a92ecce17c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:43.768949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-de080ce2-c0f1-47cb-8245-a92ecce17c1d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:43.773548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-de080ce2-c0f1-47cb-8245-a92ecce17c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:43.892706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1081/2161] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:43.908055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf5af8-00b5-4c19-bc3d-dd8d6ff47b7c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:43.908866 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf5af8-00b5-4c19-bc3d-dd8d6ff47b7c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:43.909296 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf5af8-00b5-4c19-bc3d-dd8d6ff47b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:43.935366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-74bf5af8-00b5-4c19-bc3d-dd8d6ff47b7c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:43.937421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-74bf5af8-00b5-4c19-bc3d-dd8d6ff47b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:43.949927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1081/2162] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:05:43.954970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de080ce2-c0f1-47cb-8245-a92ecce17c1d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:43.971877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de080ce2-c0f1-47cb-8245-a92ecce17c1d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:43.981023 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13ca53-4606-4f64-abe5-917d987462f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:43.982398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13ca53-4606-4f64-abe5-917d987462f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:43.982757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13ca53-4606-4f64-abe5-917d987462f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:43.983071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13ca53-4606-4f64-abe5-917d987462f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:43.983867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13ca53-4606-4f64-abe5-917d987462f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:05:43.984468 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13ca53-4606-4f64-abe5-917d987462f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:43.986896 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13ca53-4606-4f64-abe5-917d987462f3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:44.004286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1082/2163] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:05:44.009133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf5af8-00b5-4c19-bc3d-dd8d6ff47b7c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:44.019676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf5af8-00b5-4c19-bc3d-dd8d6ff47b7c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:44.027418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3cd76-c248-4458-a045-a8f9feb78830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:44.028154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3cd76-c248-4458-a045-a8f9feb78830 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:44.028439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3cd76-c248-4458-a045-a8f9feb78830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:44.028710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3cd76-c248-4458-a045-a8f9feb78830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:44.029442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3cd76-c248-4458-a045-a8f9feb78830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:44.031517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3cd76-c248-4458-a045-a8f9feb78830 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:44.073513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1082/2164] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:05:44 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:05:44.074968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13ca53-4606-4f64-abe5-917d987462f3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:44.089682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13ca53-4606-4f64-abe5-917d987462f3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:44.196271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aebee6-4309-4cf5-b64b-1718182c83df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:44.196954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aebee6-4309-4cf5-b64b-1718182c83df admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:44.197377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aebee6-4309-4cf5-b64b-1718182c83df admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:44.197699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aebee6-4309-4cf5-b64b-1718182c83df admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:44.198412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aebee6-4309-4cf5-b64b-1718182c83df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:44.200641 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aebee6-4309-4cf5-b64b-1718182c83df admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:44.235056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1083/2165] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:05:44 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:05:44.239858 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3cd76-c248-4458-a045-a8f9feb78830 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:44.251362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3cd76-c248-4458-a045-a8f9feb78830 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:44.256961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da794e00-514d-4500-b692-3b43ab3f8113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:44.257696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da794e00-514d-4500-b692-3b43ab3f8113 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:44.258162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da794e00-514d-4500-b692-3b43ab3f8113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:44.258273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da794e00-514d-4500-b692-3b43ab3f8113 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:44.258997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da794e00-514d-4500-b692-3b43ab3f8113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:44.260853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da794e00-514d-4500-b692-3b43ab3f8113 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:44.280600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1083/2166] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:44.285027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aebee6-4309-4cf5-b64b-1718182c83df admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:44.301246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aebee6-4309-4cf5-b64b-1718182c83df admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:44.308286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd2024-bc3e-444b-86ed-aa28606b6c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:44.309111 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd2024-bc3e-444b-86ed-aa28606b6c42 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:44.309406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd2024-bc3e-444b-86ed-aa28606b6c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:44.309852 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd2024-bc3e-444b-86ed-aa28606b6c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:44.310625 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd2024-bc3e-444b-86ed-aa28606b6c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:44.312591 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd2024-bc3e-444b-86ed-aa28606b6c42 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:44.327672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1084/2167] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:05:44.333423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da794e00-514d-4500-b692-3b43ab3f8113 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:44.341828 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:44.347693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da794e00-514d-4500-b692-3b43ab3f8113 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:44.362340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732994d5-e721-458c-95a1-d366c3f2ee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:44.363389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732994d5-e721-458c-95a1-d366c3f2ee70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:44.363765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732994d5-e721-458c-95a1-d366c3f2ee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:44.364058 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732994d5-e721-458c-95a1-d366c3f2ee70 admin admin] PATH_INFO: `/v3/projects/9867d9b808ad4d21be3084af9c2499fb/users/e330cca6f3dd4865ae63da0acdeb4848/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:44.385364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732994d5-e721-458c-95a1-d366c3f2ee70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9867d9b808ad4d21be3084af9c2499fb, user_id=e330cca6f3dd4865ae63da0acdeb4848, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:44.388518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732994d5-e721-458c-95a1-d366c3f2ee70 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:44.418198 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1084/2168] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:05:44 2022] PUT /identity/v3/projects/9867d9b808ad4d21be3084af9c2499fb/users/e330cca6f3dd4865ae63da0acdeb4848/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:05:44.433049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e24f8b-ed31-4a90-828a-69a78efc562f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:44.433736 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e24f8b-ed31-4a90-828a-69a78efc562f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:44.434404 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e24f8b-ed31-4a90-828a-69a78efc562f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:44.492762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-58e24f8b-ed31-4a90-828a-69a78efc562f None None] MFA Rules not processed for user `e330cca6f3dd4865ae63da0acdeb4848`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:44.494875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-58e24f8b-ed31-4a90-828a-69a78efc562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:44.594807 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:44.658036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1085/2169] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:44.669560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732994d5-e721-458c-95a1-d366c3f2ee70 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:44.673700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-732994d5-e721-458c-95a1-d366c3f2ee70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:44.819509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732994d5-e721-458c-95a1-d366c3f2ee70 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:44.826365 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743509c-1a9f-4390-8d34-f9226f46c00a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:44.827191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743509c-1a9f-4390-8d34-f9226f46c00a service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:44.827694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743509c-1a9f-4390-8d34-f9226f46c00a service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:44.828072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743509c-1a9f-4390-8d34-f9226f46c00a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:44.842086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743509c-1a9f-4390-8d34-f9226f46c00a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:44.843938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743509c-1a9f-4390-8d34-f9226f46c00a service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:44.878927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1085/2170] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:53.462679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5b698-c477-4b71-9695-d9b871ad0e35 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:53.463176 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5b698-c477-4b71-9695-d9b871ad0e35 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:53.463176 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5b698-c477-4b71-9695-d9b871ad0e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:53.536288 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-68d5b698-c477-4b71-9695-d9b871ad0e35 None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:53.538824 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-68d5b698-c477-4b71-9695-d9b871ad0e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:53.704428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1086/2171] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:05:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:53.769469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743509c-1a9f-4390-8d34-f9226f46c00a service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:53.770992 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e743509c-1a9f-4390-8d34-f9226f46c00a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:53.956001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743509c-1a9f-4390-8d34-f9226f46c00a service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:53.961614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3110f36-602e-43c2-9575-d8ca7f1b193e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:53.962315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3110f36-602e-43c2-9575-d8ca7f1b193e service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:53.962605 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3110f36-602e-43c2-9575-d8ca7f1b193e service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:53.962881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3110f36-602e-43c2-9575-d8ca7f1b193e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:53.977750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3110f36-602e-43c2-9575-d8ca7f1b193e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:53.981276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3110f36-602e-43c2-9575-d8ca7f1b193e service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:54.012210 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1086/2172] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:05:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 08:05:54.942870 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fdbe9-86cb-4a0b-8d76-733584c37eaf None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:54.943560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fdbe9-86cb-4a0b-8d76-733584c37eaf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:54.943943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fdbe9-86cb-4a0b-8d76-733584c37eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:54.978903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-036fdbe9-86cb-4a0b-8d76-733584c37eaf None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:54.981091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-036fdbe9-86cb-4a0b-8d76-733584c37eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:54.993453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1087/2173] 10.222.0.43 () {58 vars in 967 bytes} [Mon Jul 4 08:05:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:55.014885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3110f36-602e-43c2-9575-d8ca7f1b193e service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:55.028359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3110f36-602e-43c2-9575-d8ca7f1b193e service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:55.035954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9ba55-9b95-4341-89f3-10753e55e6a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:55.036717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9ba55-9b95-4341-89f3-10753e55e6a6 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:55.036934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9ba55-9b95-4341-89f3-10753e55e6a6 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:55.037268 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9ba55-9b95-4341-89f3-10753e55e6a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:55.052089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9ba55-9b95-4341-89f3-10753e55e6a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:55.055392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9ba55-9b95-4341-89f3-10753e55e6a6 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:55.073817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1087/2174] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:05:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 08:05:55.430984 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b3e45-9587-48ad-adcd-6d9e0ce43a7a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:55.432343 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b3e45-9587-48ad-adcd-6d9e0ce43a7a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:55.432511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b3e45-9587-48ad-adcd-6d9e0ce43a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:55.461577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-684b3e45-9587-48ad-adcd-6d9e0ce43a7a None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:55.463465 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-684b3e45-9587-48ad-adcd-6d9e0ce43a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:55.498282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1088/2175] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:05:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:55.532002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9ba55-9b95-4341-89f3-10753e55e6a6 service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:55.552060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9ba55-9b95-4341-89f3-10753e55e6a6 service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:55.562613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bc2f0-2231-4fe0-840b-a30f8d204fba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:55.564157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bc2f0-2231-4fe0-840b-a30f8d204fba service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:55.564907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bc2f0-2231-4fe0-840b-a30f8d204fba service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:55.565659 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bc2f0-2231-4fe0-840b-a30f8d204fba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:55.585194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bc2f0-2231-4fe0-840b-a30f8d204fba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:55.588278 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bc2f0-2231-4fe0-840b-a30f8d204fba service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:55.608935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1088/2176] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:05:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 08:05:56.093834 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad175f-783d-43ee-84e2-64c35acea7d8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:56.094677 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad175f-783d-43ee-84e2-64c35acea7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:56.095580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad175f-783d-43ee-84e2-64c35acea7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:56.142327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-19ad175f-783d-43ee-84e2-64c35acea7d8 None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:56.146200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-19ad175f-783d-43ee-84e2-64c35acea7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:56.163559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1089/2177] 10.222.0.43 () {58 vars in 967 bytes} [Mon Jul 4 08:05:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:56.182346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bc2f0-2231-4fe0-840b-a30f8d204fba service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:56.212357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bc2f0-2231-4fe0-840b-a30f8d204fba service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:56.217998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84e505-3402-4424-9919-60dfe504d308 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:56.219425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84e505-3402-4424-9919-60dfe504d308 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:56.220089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84e505-3402-4424-9919-60dfe504d308 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:56.220841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84e505-3402-4424-9919-60dfe504d308 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:56.239744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84e505-3402-4424-9919-60dfe504d308 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:56.242652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84e505-3402-4424-9919-60dfe504d308 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:56.261813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1089/2178] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:05:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 08:05:58.177345 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad175f-783d-43ee-84e2-64c35acea7d8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:58.304995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad175f-783d-43ee-84e2-64c35acea7d8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:58.313511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:58.314793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:58.315498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:58.316112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] PATH_INFO: `/v3/users/e330cca6f3dd4865ae63da0acdeb4848` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:58.320245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e330cca6f3dd4865ae63da0acdeb4848)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:58.323455 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:58.357054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:58.401288 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e330cca6f3dd4865ae63da0acdeb4848'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:58.427773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e330cca6f3dd4865ae63da0acdeb4848'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:58.448622 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e330cca6f3dd4865ae63da0acdeb4848'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:58.450955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e330cca6f3dd4865ae63da0acdeb4848'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:58.475741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e330cca6f3dd4865ae63da0acdeb4848'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:58.488001 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1090/2179] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:58 2022] DELETE /identity/v3/users/e330cca6f3dd4865ae63da0acdeb4848 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:58.496102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547cc1a-6d71-4765-bb03-4a913b0e37bb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:58.496850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547cc1a-6d71-4765-bb03-4a913b0e37bb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:58.497089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547cc1a-6d71-4765-bb03-4a913b0e37bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:58.507009 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:58.559864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c547cc1a-6d71-4765-bb03-4a913b0e37bb None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:58.562159 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c547cc1a-6d71-4765-bb03-4a913b0e37bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:58.689908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1090/2180] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:58.707147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:58.710214 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:58.890159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c2ee7-770d-4f27-b5eb-a34cc8612b3c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:58.903064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81508c4d-8ef3-405a-878d-67e095af1ccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:58.904148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508c4d-8ef3-405a-878d-67e095af1ccd service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:58.904630 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508c4d-8ef3-405a-878d-67e095af1ccd service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:58.905121 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508c4d-8ef3-405a-878d-67e095af1ccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:58.923410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81508c4d-8ef3-405a-878d-67e095af1ccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:58.926489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81508c4d-8ef3-405a-878d-67e095af1ccd service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:58.947701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1091/2181] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:59.006529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547cc1a-6d71-4765-bb03-4a913b0e37bb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:59.009570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c547cc1a-6d71-4765-bb03-4a913b0e37bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:59.038740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547cc1a-6d71-4765-bb03-4a913b0e37bb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:59.050401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:59.052258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:59.052771 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:59.053310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] PATH_INFO: `/v3/users/1fd15df599fa4261b917f4f9b1d1f75a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:59.056026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd15df599fa4261b917f4f9b1d1f75a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:59.059971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:59.200480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd15df599fa4261b917f4f9b1d1f75a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:59.203305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd15df599fa4261b917f4f9b1d1f75a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:59.204551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd15df599fa4261b917f4f9b1d1f75a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:59.217479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd15df599fa4261b917f4f9b1d1f75a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:59.246022 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd15df599fa4261b917f4f9b1d1f75a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:59.286302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1091/2182] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:05:59 2022] DELETE /identity/v3/users/1fd15df599fa4261b917f4f9b1d1f75a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:05:59.290970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a732bc-fae5-4da5-967f-349696808723 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:59.291465 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a732bc-fae5-4da5-967f-349696808723 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:59.291986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a732bc-fae5-4da5-967f-349696808723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:59.352782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-70a732bc-fae5-4da5-967f-349696808723 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:05:59.355051 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-70a732bc-fae5-4da5-967f-349696808723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:59.545382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:59.547760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:59.600587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1092/2183] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:05:59.615772 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a732bc-fae5-4da5-967f-349696808723 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:05:59.617895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-70a732bc-fae5-4da5-967f-349696808723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:05:59.683444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:59.689447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23920a0-91bb-465d-9b30-8989ab2bae0d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:59.703842 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096bf90-828f-49f7-8f92-bb8e05f95e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:59.706040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096bf90-828f-49f7-8f92-bb8e05f95e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:59.706791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096bf90-828f-49f7-8f92-bb8e05f95e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:59.707305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096bf90-828f-49f7-8f92-bb8e05f95e54 admin admin] PATH_INFO: `/v3/projects/9867d9b808ad4d21be3084af9c2499fb` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:59.709175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7096bf90-828f-49f7-8f92-bb8e05f95e54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9867d9b808ad4d21be3084af9c2499fb)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:59.722330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7096bf90-828f-49f7-8f92-bb8e05f95e54 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:59.733262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:05:59.795991 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a732bc-fae5-4da5-967f-349696808723 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:05:59.801012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f751c-06d5-46f0-b06a-1134105f63db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:05:59.802031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f751c-06d5-46f0-b06a-1134105f63db service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:59.802908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f751c-06d5-46f0-b06a-1134105f63db service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:59.802908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f751c-06d5-46f0-b06a-1134105f63db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:05:59.827702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f751c-06d5-46f0-b06a-1134105f63db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:05:59.830636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f751c-06d5-46f0-b06a-1134105f63db service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:05:59.856759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1093/2184] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:05:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:05:59.888743 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7096bf90-828f-49f7-8f92-bb8e05f95e54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9867d9b808ad4d21be3084af9c2499fb', 'request_id': 'req-7096bf90-828f-49f7-8f92-bb8e05f95e54'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:59.898686 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7096bf90-828f-49f7-8f92-bb8e05f95e54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9867d9b808ad4d21be3084af9c2499fb', 'request_id': 'req-7096bf90-828f-49f7-8f92-bb8e05f95e54'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:59.919346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7096bf90-828f-49f7-8f92-bb8e05f95e54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9867d9b808ad4d21be3084af9c2499fb', 'request_id': 'req-7096bf90-828f-49f7-8f92-bb8e05f95e54'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:59.957106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7096bf90-828f-49f7-8f92-bb8e05f95e54 admin admin] The token cache is being invalidate because project 9867d9b808ad4d21be3084af9c2499fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:05:59.958900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7096bf90-828f-49f7-8f92-bb8e05f95e54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:59.959779 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7096bf90-828f-49f7-8f92-bb8e05f95e54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:05:59.963125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1092/2185] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:05:59 2022] DELETE /identity/v3/projects/9867d9b808ad4d21be3084af9c2499fb => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:05:59.971382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2c842-5cea-44d7-85b9-26e321724ad7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:05:59.972042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2c842-5cea-44d7-85b9-26e321724ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:05:59.972397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2c842-5cea-44d7-85b9-26e321724ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.051064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bac2c842-5cea-44d7-85b9-26e321724ad7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:00.052044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bac2c842-5cea-44d7-85b9-26e321724ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:00.182154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1094/2186] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:00.197566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715fce7-d344-48a4-a1e8-76f27a8f0b89 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.198041 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715fce7-d344-48a4-a1e8-76f27a8f0b89 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.198436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715fce7-d344-48a4-a1e8-76f27a8f0b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.229428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5715fce7-d344-48a4-a1e8-76f27a8f0b89 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:00.230789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5715fce7-d344-48a4-a1e8-76f27a8f0b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:00.250821 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1093/2187] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:00.254390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2c842-5cea-44d7-85b9-26e321724ad7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:00.261449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715fce7-d344-48a4-a1e8-76f27a8f0b89 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:00.265384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5715fce7-d344-48a4-a1e8-76f27a8f0b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:00.266536 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2c842-5cea-44d7-85b9-26e321724ad7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:00.273762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f446c-f3d4-4563-be59-3bf96be7c8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:00.275011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f446c-f3d4-4563-be59-3bf96be7c8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.276050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f446c-f3d4-4563-be59-3bf96be7c8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.276634 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f446c-f3d4-4563-be59-3bf96be7c8aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.277485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f446c-f3d4-4563-be59-3bf96be7c8aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:06:00.277982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f446c-f3d4-4563-be59-3bf96be7c8aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:00.280232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f446c-f3d4-4563-be59-3bf96be7c8aa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:00.293045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715fce7-d344-48a4-a1e8-76f27a8f0b89 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:00.304156 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:00.305246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1095/2188] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:06:00.308982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.309354 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.309742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd admin admin] PATH_INFO: `/v3/projects/753d5874ae814841af6314a4db696221` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.310753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f446c-f3d4-4563-be59-3bf96be7c8aa admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:00.311317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=753d5874ae814841af6314a4db696221)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:00.313436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:00.327731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f446c-f3d4-4563-be59-3bf96be7c8aa admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:00.333959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7805342-f185-4aed-b6c5-6e52e6764e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:00.334756 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7805342-f185-4aed-b6c5-6e52e6764e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.335272 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7805342-f185-4aed-b6c5-6e52e6764e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.335739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7805342-f185-4aed-b6c5-6e52e6764e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.336749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7805342-f185-4aed-b6c5-6e52e6764e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:00.339229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7805342-f185-4aed-b6c5-6e52e6764e93 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:00.385577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1096/2189] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:06:00 2022] POST /identity/v3/projects => generated 389 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:06:00.385885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '753d5874ae814841af6314a4db696221', 'request_id': 'req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:00.387487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7805342-f185-4aed-b6c5-6e52e6764e93 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:00.393985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '753d5874ae814841af6314a4db696221', 'request_id': 'req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:00.408669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '753d5874ae814841af6314a4db696221', 'request_id': 'req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:00.435120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd admin admin] The token cache is being invalidate because project 753d5874ae814841af6314a4db696221 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:06:00.435753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:00.436850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7333eebb-dfc9-41e1-a3fc-1ec25b441dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:00.439495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1094/2190] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:06:00 2022] DELETE /identity/v3/projects/753d5874ae814841af6314a4db696221 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:06:00.445735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6606bc0-b4f7-4613-bc88-2f615af8d247 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.446158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6606bc0-b4f7-4613-bc88-2f615af8d247 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.446547 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6606bc0-b4f7-4613-bc88-2f615af8d247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.470479 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c6606bc0-b4f7-4613-bc88-2f615af8d247 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:00.471847 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c6606bc0-b4f7-4613-bc88-2f615af8d247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:00.482309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1095/2191] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:00.489442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7805342-f185-4aed-b6c5-6e52e6764e93 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:00.491595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42859bb-95df-43a7-a11f-6414327a303c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.492009 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42859bb-95df-43a7-a11f-6414327a303c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.492330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42859bb-95df-43a7-a11f-6414327a303c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.498479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfc5fd-d9a7-4828-87d1-bfe4f85a6e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:00.499468 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfc5fd-d9a7-4828-87d1-bfe4f85a6e50 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.499792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfc5fd-d9a7-4828-87d1-bfe4f85a6e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.500017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfc5fd-d9a7-4828-87d1-bfe4f85a6e50 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.500762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfc5fd-d9a7-4828-87d1-bfe4f85a6e50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:00.503862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfc5fd-d9a7-4828-87d1-bfe4f85a6e50 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:00.515364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f42859bb-95df-43a7-a11f-6414327a303c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:00.516644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f42859bb-95df-43a7-a11f-6414327a303c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:00.528126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1096/2192] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:06:00.533178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42859bb-95df-43a7-a11f-6414327a303c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:00.543702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1097/2193] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:06:00 2022] POST /identity/v3/users => generated 295 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:06:00.547116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42859bb-95df-43a7-a11f-6414327a303c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:00.549381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfc5fd-d9a7-4828-87d1-bfe4f85a6e50 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:00.551355 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a3cfc5fd-d9a7-4828-87d1-bfe4f85a6e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:00.555070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e03e0-93d6-4ab9-88e5-8700119f42f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:00.556097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e03e0-93d6-4ab9-88e5-8700119f42f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.556710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e03e0-93d6-4ab9-88e5-8700119f42f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.557233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e03e0-93d6-4ab9-88e5-8700119f42f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.558306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e03e0-93d6-4ab9-88e5-8700119f42f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:06:00.558755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e03e0-93d6-4ab9-88e5-8700119f42f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:00.561655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e03e0-93d6-4ab9-88e5-8700119f42f5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:00.567281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfc5fd-d9a7-4828-87d1-bfe4f85a6e50 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:00.572872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc0365-e819-4e6d-aded-d88dae27d11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:00.573752 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc0365-e819-4e6d-aded-d88dae27d11d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.574162 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc0365-e819-4e6d-aded-d88dae27d11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.574771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc0365-e819-4e6d-aded-d88dae27d11d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.575799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc0365-e819-4e6d-aded-d88dae27d11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:00.577815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc0365-e819-4e6d-aded-d88dae27d11d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:00.579540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:00.584364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1097/2194] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:06:00.591619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e03e0-93d6-4ab9-88e5-8700119f42f5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:00.598274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1098/2195] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:06:00.604999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc0365-e819-4e6d-aded-d88dae27d11d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:00.609682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e03e0-93d6-4ab9-88e5-8700119f42f5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:00.618024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9831f-e73a-4c1b-bf3d-98e1185e9945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:00.619156 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9831f-e73a-4c1b-bf3d-98e1185e9945 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.619691 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9831f-e73a-4c1b-bf3d-98e1185e9945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.620200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9831f-e73a-4c1b-bf3d-98e1185e9945 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.621537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9831f-e73a-4c1b-bf3d-98e1185e9945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:00.623053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc0365-e819-4e6d-aded-d88dae27d11d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:00.625442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9831f-e73a-4c1b-bf3d-98e1185e9945 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:00.634418 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de58580-1a9e-4989-bd4b-1aa868003a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:00.635463 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de58580-1a9e-4989-bd4b-1aa868003a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.635911 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de58580-1a9e-4989-bd4b-1aa868003a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.636390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de58580-1a9e-4989-bd4b-1aa868003a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.637848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de58580-1a9e-4989-bd4b-1aa868003a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:00.640692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de58580-1a9e-4989-bd4b-1aa868003a0c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:00.660833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1099/2196] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:06:00.667319 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de58580-1a9e-4989-bd4b-1aa868003a0c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:00.677994 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1098/2197] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:06:00 2022] POST /identity/v3/projects => generated 371 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:06:00.684913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9831f-e73a-4c1b-bf3d-98e1185e9945 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:00.687416 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de58580-1a9e-4989-bd4b-1aa868003a0c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:00.721589 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:00.800649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223bafcc-54a3-44f6-b599-9fa58c90a104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:00.801968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bafcc-54a3-44f6-b599-9fa58c90a104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.802251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bafcc-54a3-44f6-b599-9fa58c90a104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.802573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bafcc-54a3-44f6-b599-9fa58c90a104 admin admin] PATH_INFO: `/v3/projects/3db22c79ad1245c7abff313384460525/users/519e6300514645e7823f68d733152c40/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:00.823068 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223bafcc-54a3-44f6-b599-9fa58c90a104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3db22c79ad1245c7abff313384460525, user_id=519e6300514645e7823f68d733152c40, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:00.826155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223bafcc-54a3-44f6-b599-9fa58c90a104 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:00.859312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9831f-e73a-4c1b-bf3d-98e1185e9945 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:00.866716 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1100/2198] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:06:00 2022] PUT /identity/v3/projects/3db22c79ad1245c7abff313384460525/users/519e6300514645e7823f68d733152c40/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:06:00.903919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc551e09-49d6-4564-bfc4-3cfc22848d27 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:00.905058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc551e09-49d6-4564-bfc4-3cfc22848d27 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:00.906588 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc551e09-49d6-4564-bfc4-3cfc22848d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:01.038208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cc551e09-49d6-4564-bfc4-3cfc22848d27 None None] MFA Rules not processed for user `519e6300514645e7823f68d733152c40`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:01.041389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cc551e09-49d6-4564-bfc4-3cfc22848d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:01.069141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd24aa-581b-47fd-8bef-0950c9148ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:01.071710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd24aa-581b-47fd-8bef-0950c9148ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:01.074207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd24aa-581b-47fd-8bef-0950c9148ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:01.075033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd24aa-581b-47fd-8bef-0950c9148ed9 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:01.076988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd24aa-581b-47fd-8bef-0950c9148ed9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:01.081108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd24aa-581b-47fd-8bef-0950c9148ed9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:01.169840 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1099/2199] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:06:00 2022] POST /identity/v3/users => generated 286 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:06:01.172376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ec1b8-ba38-43b0-b5b1-2d81d6dcae2b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:01.172887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ec1b8-ba38-43b0-b5b1-2d81d6dcae2b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:01.174047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ec1b8-ba38-43b0-b5b1-2d81d6dcae2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:01.242290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1101/2200] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:01.246438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc551e09-49d6-4564-bfc4-3cfc22848d27 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:01.272116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-410ec1b8-ba38-43b0-b5b1-2d81d6dcae2b None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:01.277352 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-410ec1b8-ba38-43b0-b5b1-2d81d6dcae2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:01.285157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc551e09-49d6-4564-bfc4-3cfc22848d27 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:01.298221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555bcf6-61aa-45d9-9853-9eb751c671e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:01.298937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555bcf6-61aa-45d9-9853-9eb751c671e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:01.299220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555bcf6-61aa-45d9-9853-9eb751c671e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:01.299501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555bcf6-61aa-45d9-9853-9eb751c671e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:01.300301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555bcf6-61aa-45d9-9853-9eb751c671e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:01.302062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555bcf6-61aa-45d9-9853-9eb751c671e9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:01.325719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1102/2201] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:06:01.327349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555bcf6-61aa-45d9-9853-9eb751c671e9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:01.331089 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f555bcf6-61aa-45d9-9853-9eb751c671e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:01.386918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1100/2202] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:06:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:01.388150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ec1b8-ba38-43b0-b5b1-2d81d6dcae2b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:01.404845 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ec1b8-ba38-43b0-b5b1-2d81d6dcae2b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:01.412021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e2d0f-d8c6-43e7-bf47-0ac518ae910b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:01.413393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2d0f-d8c6-43e7-bf47-0ac518ae910b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:01.414265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2d0f-d8c6-43e7-bf47-0ac518ae910b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:01.414776 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2d0f-d8c6-43e7-bf47-0ac518ae910b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:01.415789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e2d0f-d8c6-43e7-bf47-0ac518ae910b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:01.429173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e2d0f-d8c6-43e7-bf47-0ac518ae910b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:01.448023 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:01.453231 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1101/2203] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:06:01.455297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e2d0f-d8c6-43e7-bf47-0ac518ae910b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:01.459103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-374e2d0f-d8c6-43e7-bf47-0ac518ae910b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:01.462843 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555bcf6-61aa-45d9-9853-9eb751c671e9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:01.475582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0a547-2791-4e78-94c6-2fb739614923 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:01.476280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0a547-2791-4e78-94c6-2fb739614923 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:01.476642 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0a547-2791-4e78-94c6-2fb739614923 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:01.476972 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0a547-2791-4e78-94c6-2fb739614923 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:01.497665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0a547-2791-4e78-94c6-2fb739614923 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:01.499166 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0a547-2791-4e78-94c6-2fb739614923 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:01.515990 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1103/2204] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:06:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:06:01.517236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0a547-2791-4e78-94c6-2fb739614923 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:01.533444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0a547-2791-4e78-94c6-2fb739614923 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:01.539757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302ad6e-ec1a-4338-a2e7-412f4c9a380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:01.540873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302ad6e-ec1a-4338-a2e7-412f4c9a380b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:01.542065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302ad6e-ec1a-4338-a2e7-412f4c9a380b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:01.542506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302ad6e-ec1a-4338-a2e7-412f4c9a380b admin admin] PATH_INFO: `/v3/projects/2609f0b160e0450798be03f7d3e993ad/users/a860176540c64e6db269f9ebe4d75753/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:01.565577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302ad6e-ec1a-4338-a2e7-412f4c9a380b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2609f0b160e0450798be03f7d3e993ad, user_id=a860176540c64e6db269f9ebe4d75753, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:01.571411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302ad6e-ec1a-4338-a2e7-412f4c9a380b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:01.596067 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:01.610597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1104/2205] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:06:01 2022] PUT /identity/v3/projects/2609f0b160e0450798be03f7d3e993ad/users/a860176540c64e6db269f9ebe4d75753/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:06:01.617775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23376e40-18ee-4c99-a91a-29b8f7c17bfc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:01.618402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23376e40-18ee-4c99-a91a-29b8f7c17bfc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:01.618709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23376e40-18ee-4c99-a91a-29b8f7c17bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:01.639833 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e2d0f-d8c6-43e7-bf47-0ac518ae910b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:01.658845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-23376e40-18ee-4c99-a91a-29b8f7c17bfc None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:01.660468 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-23376e40-18ee-4c99-a91a-29b8f7c17bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:01.757840 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59744a2-ef9a-4f82-883f-0f93401a9475 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:01.759086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59744a2-ef9a-4f82-883f-0f93401a9475 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:01.759495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59744a2-ef9a-4f82-883f-0f93401a9475 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:01.759890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59744a2-ef9a-4f82-883f-0f93401a9475 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:01.785346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1105/2206] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:06:01.803194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23376e40-18ee-4c99-a91a-29b8f7c17bfc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:01.806542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-23376e40-18ee-4c99-a91a-29b8f7c17bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:01.862624 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59744a2-ef9a-4f82-883f-0f93401a9475 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:01.864444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59744a2-ef9a-4f82-883f-0f93401a9475 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:01.876947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1102/2207] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:06:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 08:06:01.916139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23376e40-18ee-4c99-a91a-29b8f7c17bfc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:01.921884 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187806b-8d81-4125-8bda-5378187a18ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:01.922860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187806b-8d81-4125-8bda-5378187a18ea service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:01.924357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187806b-8d81-4125-8bda-5378187a18ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:01.924357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187806b-8d81-4125-8bda-5378187a18ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:01.941672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7187806b-8d81-4125-8bda-5378187a18ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:01.947637 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7187806b-8d81-4125-8bda-5378187a18ea service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:01.984232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1106/2208] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:06:11.871744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=59 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:11.956894 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=68,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:12.283069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc762e-f3e7-4132-8baf-5eeca4354a72 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:12.283069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc762e-f3e7-4132-8baf-5eeca4354a72 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:12.283490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc762e-f3e7-4132-8baf-5eeca4354a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:12.390789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e8dc762e-f3e7-4132-8baf-5eeca4354a72 None None] MFA Rules not processed for user `a860176540c64e6db269f9ebe4d75753`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:12.392818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e8dc762e-f3e7-4132-8baf-5eeca4354a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:12.534712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1103/2209] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:12.535409 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187806b-8d81-4125-8bda-5378187a18ea service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:12.554478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187806b-8d81-4125-8bda-5378187a18ea service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:12.560528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1facd-e345-423a-a61b-7caca308fab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:12.561495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1facd-e345-423a-a61b-7caca308fab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:12.562003 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1facd-e345-423a-a61b-7caca308fab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:12.562435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1facd-e345-423a-a61b-7caca308fab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:12.564281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1facd-e345-423a-a61b-7caca308fab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:12.566583 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1facd-e345-423a-a61b-7caca308fab8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:12.631124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1107/2210] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:06:12 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:06:12.637410 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc762e-f3e7-4132-8baf-5eeca4354a72 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:12.653129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc762e-f3e7-4132-8baf-5eeca4354a72 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:12.750731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb28d7f-e36f-45aa-8ac0-d20a77a4d174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:12.751721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb28d7f-e36f-45aa-8ac0-d20a77a4d174 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:12.752562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb28d7f-e36f-45aa-8ac0-d20a77a4d174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:12.752851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb28d7f-e36f-45aa-8ac0-d20a77a4d174 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:12.753578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb28d7f-e36f-45aa-8ac0-d20a77a4d174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:12.755766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb28d7f-e36f-45aa-8ac0-d20a77a4d174 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:12.784956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1104/2211] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:06:12 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:06:12.786491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1facd-e345-423a-a61b-7caca308fab8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:12.800878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1facd-e345-423a-a61b-7caca308fab8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:12.808686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e041711-5213-4653-a24a-98a9197bcc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:12.809411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e041711-5213-4653-a24a-98a9197bcc0f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:12.809768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e041711-5213-4653-a24a-98a9197bcc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:12.810057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e041711-5213-4653-a24a-98a9197bcc0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:12.824955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e041711-5213-4653-a24a-98a9197bcc0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:12.827005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e041711-5213-4653-a24a-98a9197bcc0f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:12.857350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1108/2212] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:06:12.859185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb28d7f-e36f-45aa-8ac0-d20a77a4d174 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:12.879242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb28d7f-e36f-45aa-8ac0-d20a77a4d174 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:12.899103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027473af-86f3-40ec-a456-4959023569e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:12.905719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027473af-86f3-40ec-a456-4959023569e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:12.906433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027473af-86f3-40ec-a456-4959023569e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:12.906997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027473af-86f3-40ec-a456-4959023569e7 admin admin] PATH_INFO: `/v3/projects/2f59148d214f4b94a9f9787dc1068775/users/7b3c639531b14f06ba93f30ee5c7a7c1/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:12.972925 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027473af-86f3-40ec-a456-4959023569e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f59148d214f4b94a9f9787dc1068775, user_id=7b3c639531b14f06ba93f30ee5c7a7c1, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:12.981386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027473af-86f3-40ec-a456-4959023569e7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:13.006376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1105/2213] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:06:12 2022] PUT /identity/v3/projects/2f59148d214f4b94a9f9787dc1068775/users/7b3c639531b14f06ba93f30ee5c7a7c1/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:06:13.010722 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e041711-5213-4653-a24a-98a9197bcc0f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:13.030093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e041711-5213-4653-a24a-98a9197bcc0f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:13.276033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45736e-3bdd-42b2-8131-406222956d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:13.276857 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45736e-3bdd-42b2-8131-406222956d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:13.277141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45736e-3bdd-42b2-8131-406222956d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:13.277421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45736e-3bdd-42b2-8131-406222956d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:13.279592 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45736e-3bdd-42b2-8131-406222956d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:13.279949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45736e-3bdd-42b2-8131-406222956d54 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:13.308367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1109/2214] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:06:13.323363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027473af-86f3-40ec-a456-4959023569e7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:13.359056 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027473af-86f3-40ec-a456-4959023569e7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:13.371047 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33a847-c7bf-4faf-abce-e97bee8289cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:13.372643 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33a847-c7bf-4faf-abce-e97bee8289cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:13.373126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33a847-c7bf-4faf-abce-e97bee8289cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:13.373770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33a847-c7bf-4faf-abce-e97bee8289cd admin admin] PATH_INFO: `/v3/projects/2f59148d214f4b94a9f9787dc1068775/users/7b3c639531b14f06ba93f30ee5c7a7c1/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:13.377536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33a847-c7bf-4faf-abce-e97bee8289cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f59148d214f4b94a9f9787dc1068775, user_id=7b3c639531b14f06ba93f30ee5c7a7c1, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:13.380632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33a847-c7bf-4faf-abce-e97bee8289cd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:13.418272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1106/2215] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:06:13 2022] PUT /identity/v3/projects/2f59148d214f4b94a9f9787dc1068775/users/7b3c639531b14f06ba93f30ee5c7a7c1/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:06:21.457487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa4b04-11ae-4423-a81a-552a57081fcd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:21.458074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa4b04-11ae-4423-a81a-552a57081fcd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:21.458074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa4b04-11ae-4423-a81a-552a57081fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:21.498893 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c6fa4b04-11ae-4423-a81a-552a57081fcd None None] MFA Rules not processed for user `7b3c639531b14f06ba93f30ee5c7a7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:21.500392 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c6fa4b04-11ae-4423-a81a-552a57081fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:21.655732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1110/2216] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:21 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:21.671995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33a847-c7bf-4faf-abce-e97bee8289cd admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:21.674117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cc33a847-c7bf-4faf-abce-e97bee8289cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:21.731682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:21.806772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33a847-c7bf-4faf-abce-e97bee8289cd admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:21.812486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffc237-e8d3-4e57-860c-640f2b737aac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:21.813204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffc237-e8d3-4e57-860c-640f2b737aac service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:21.813492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffc237-e8d3-4e57-860c-640f2b737aac service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:21.813804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffc237-e8d3-4e57-860c-640f2b737aac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:21.888021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ffc237-e8d3-4e57-860c-640f2b737aac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:21.889772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ffc237-e8d3-4e57-860c-640f2b737aac service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:21.904701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1107/2217] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:06:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:06:23.323908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685bb0e-58bf-4973-9061-6d22aa4e4b46 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:23.323908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685bb0e-58bf-4973-9061-6d22aa4e4b46 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:23.323908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685bb0e-58bf-4973-9061-6d22aa4e4b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:23.344421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8685bb0e-58bf-4973-9061-6d22aa4e4b46 None None] MFA Rules not processed for user `a860176540c64e6db269f9ebe4d75753`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:23.345766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8685bb0e-58bf-4973-9061-6d22aa4e4b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:23.353807 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1111/2218] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:26.863169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffc237-e8d3-4e57-860c-640f2b737aac service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:26.934965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffc237-e8d3-4e57-860c-640f2b737aac service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:26.942216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:26.943141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:26.943482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:26.943702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] PATH_INFO: `/v3/users/519e6300514645e7823f68d733152c40` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:26.945226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=519e6300514645e7823f68d733152c40)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:26.947222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:27.009478 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519e6300514645e7823f68d733152c40'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:27.010190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519e6300514645e7823f68d733152c40'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:27.010788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519e6300514645e7823f68d733152c40'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:27.016358 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:27.019094 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519e6300514645e7823f68d733152c40'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:27.032391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519e6300514645e7823f68d733152c40'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:27.048286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1108/2219] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:06:26 2022] DELETE /identity/v3/users/519e6300514645e7823f68d733152c40 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:06:27.053120 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8239d28-34b9-4fda-93eb-7409a59f1fbd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:27.053758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8239d28-34b9-4fda-93eb-7409a59f1fbd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:27.053758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8239d28-34b9-4fda-93eb-7409a59f1fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:27.072955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b8239d28-34b9-4fda-93eb-7409a59f1fbd None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:27.075232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b8239d28-34b9-4fda-93eb-7409a59f1fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:27.116393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:27.141889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1112/2220] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:27.152113 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:27.153809 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:27.216064 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c80cb9-58de-42bf-8058-d2324b0c1a94 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:27.220261 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af7864-89e5-4147-8b9f-fab5e8e4bc42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:27.220928 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af7864-89e5-4147-8b9f-fab5e8e4bc42 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:27.221213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af7864-89e5-4147-8b9f-fab5e8e4bc42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:27.221487 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af7864-89e5-4147-8b9f-fab5e8e4bc42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:27.231252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af7864-89e5-4147-8b9f-fab5e8e4bc42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:27.232731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af7864-89e5-4147-8b9f-fab5e8e4bc42 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:27.242507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1109/2221] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:06:27.667896 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8239d28-34b9-4fda-93eb-7409a59f1fbd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:27.669678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b8239d28-34b9-4fda-93eb-7409a59f1fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:27.683421 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8239d28-34b9-4fda-93eb-7409a59f1fbd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:27.690789 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d857784-c5ad-4b68-8ea4-56aa80963e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:27.692149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d857784-c5ad-4b68-8ea4-56aa80963e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:27.692659 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d857784-c5ad-4b68-8ea4-56aa80963e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:27.693090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d857784-c5ad-4b68-8ea4-56aa80963e2c admin admin] PATH_INFO: `/v3/projects/3db22c79ad1245c7abff313384460525` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:27.694964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d857784-c5ad-4b68-8ea4-56aa80963e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3db22c79ad1245c7abff313384460525)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:27.696957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d857784-c5ad-4b68-8ea4-56aa80963e2c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:27.751860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5d857784-c5ad-4b68-8ea4-56aa80963e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3db22c79ad1245c7abff313384460525', 'request_id': 'req-5d857784-c5ad-4b68-8ea4-56aa80963e2c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:27.758722 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5d857784-c5ad-4b68-8ea4-56aa80963e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3db22c79ad1245c7abff313384460525', 'request_id': 'req-5d857784-c5ad-4b68-8ea4-56aa80963e2c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:27.771018 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5d857784-c5ad-4b68-8ea4-56aa80963e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3db22c79ad1245c7abff313384460525', 'request_id': 'req-5d857784-c5ad-4b68-8ea4-56aa80963e2c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:27.784400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5d857784-c5ad-4b68-8ea4-56aa80963e2c admin admin] The token cache is being invalidate because project 3db22c79ad1245c7abff313384460525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:06:27.784688 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5d857784-c5ad-4b68-8ea4-56aa80963e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:27.785344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5d857784-c5ad-4b68-8ea4-56aa80963e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:06:27.786995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1113/2222] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:06:27 2022] DELETE /identity/v3/projects/3db22c79ad1245c7abff313384460525 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:06:27.794026 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5e5e2-7fae-4eb1-9c1d-8dc5ac8daa6f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:27.794424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5e5e2-7fae-4eb1-9c1d-8dc5ac8daa6f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:27.794866 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5e5e2-7fae-4eb1-9c1d-8dc5ac8daa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:27.824036 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3ab5e5e2-7fae-4eb1-9c1d-8dc5ac8daa6f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:27.825250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3ab5e5e2-7fae-4eb1-9c1d-8dc5ac8daa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:27.891857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1110/2223] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:27.902715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec53740-5750-4df5-b964-049f98024a1d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:27.902997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec53740-5750-4df5-b964-049f98024a1d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:27.903287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec53740-5750-4df5-b964-049f98024a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:27.925804 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2ec53740-5750-4df5-b964-049f98024a1d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:27.927078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2ec53740-5750-4df5-b964-049f98024a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:27.936902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1114/2224] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:06:27.941607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5e5e2-7fae-4eb1-9c1d-8dc5ac8daa6f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:27.952127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5e5e2-7fae-4eb1-9c1d-8dc5ac8daa6f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:27.958471 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ad9e4-8d94-4123-8038-f04cdc6bab32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:27.959484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ad9e4-8d94-4123-8038-f04cdc6bab32 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:27.959484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ad9e4-8d94-4123-8038-f04cdc6bab32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:27.959688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ad9e4-8d94-4123-8038-f04cdc6bab32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:27.960407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ad9e4-8d94-4123-8038-f04cdc6bab32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:06:27.960717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ad9e4-8d94-4123-8038-f04cdc6bab32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:27.962414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ad9e4-8d94-4123-8038-f04cdc6bab32 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:27.976315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1111/2225] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:06:27.982968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec53740-5750-4df5-b964-049f98024a1d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:27.993794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec53740-5750-4df5-b964-049f98024a1d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:27.999924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63494024-a887-4276-afe6-3df9ac48aa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:28.000605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63494024-a887-4276-afe6-3df9ac48aa5f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:28.000907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63494024-a887-4276-afe6-3df9ac48aa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:28.001353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63494024-a887-4276-afe6-3df9ac48aa5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:28.002514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63494024-a887-4276-afe6-3df9ac48aa5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:28.004496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63494024-a887-4276-afe6-3df9ac48aa5f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:28.034916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1115/2226] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:06:27 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:06:28.039099 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ad9e4-8d94-4123-8038-f04cdc6bab32 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:28.051061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ad9e4-8d94-4123-8038-f04cdc6bab32 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:28.113172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:28.123606 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfef76-cd98-44df-ab48-02276fb60a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:28.124398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfef76-cd98-44df-ab48-02276fb60a66 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:28.124716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfef76-cd98-44df-ab48-02276fb60a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:28.125024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfef76-cd98-44df-ab48-02276fb60a66 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:28.126454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bfef76-cd98-44df-ab48-02276fb60a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:28.129699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bfef76-cd98-44df-ab48-02276fb60a66 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:28.156142 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1112/2227] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:06:28 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:06:28.157586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63494024-a887-4276-afe6-3df9ac48aa5f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:28.168720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63494024-a887-4276-afe6-3df9ac48aa5f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:28.179531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d576c1e-bff9-4cf2-b8a9-014a71c22cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:28.180377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d576c1e-bff9-4cf2-b8a9-014a71c22cee admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:28.180685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d576c1e-bff9-4cf2-b8a9-014a71c22cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:28.181113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d576c1e-bff9-4cf2-b8a9-014a71c22cee admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:28.181848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d576c1e-bff9-4cf2-b8a9-014a71c22cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:28.183739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d576c1e-bff9-4cf2-b8a9-014a71c22cee admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:28.196004 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1116/2228] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:06:28.204646 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfef76-cd98-44df-ab48-02276fb60a66 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:28.211700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfef76-cd98-44df-ab48-02276fb60a66 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:28.217420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0880aaf-e366-4e66-bdad-5ff5b89d349f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:28.218150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0880aaf-e366-4e66-bdad-5ff5b89d349f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:28.218423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0880aaf-e366-4e66-bdad-5ff5b89d349f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:28.218739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0880aaf-e366-4e66-bdad-5ff5b89d349f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:28.219596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0880aaf-e366-4e66-bdad-5ff5b89d349f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:28.221598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0880aaf-e366-4e66-bdad-5ff5b89d349f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:28.233449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1113/2229] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:06:28.238151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d576c1e-bff9-4cf2-b8a9-014a71c22cee admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:28.248737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d576c1e-bff9-4cf2-b8a9-014a71c22cee admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:28.254097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c6f7c-8fe6-480f-aacd-73d32fd3895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:28.255012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c6f7c-8fe6-480f-aacd-73d32fd3895b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:28.255298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c6f7c-8fe6-480f-aacd-73d32fd3895b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:28.255575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c6f7c-8fe6-480f-aacd-73d32fd3895b admin admin] PATH_INFO: `/v3/projects/d1b0f9c7bc044f6c802c8cd34da4eb0e/users/fa28f0e0b3b0488faf7db7bf7dc3854b/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:28.272992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c6f7c-8fe6-480f-aacd-73d32fd3895b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1b0f9c7bc044f6c802c8cd34da4eb0e, user_id=fa28f0e0b3b0488faf7db7bf7dc3854b, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:28.275718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c6f7c-8fe6-480f-aacd-73d32fd3895b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:28.294739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1117/2230] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:06:28 2022] PUT /identity/v3/projects/d1b0f9c7bc044f6c802c8cd34da4eb0e/users/fa28f0e0b3b0488faf7db7bf7dc3854b/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:06:28.303340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dbeb5-47d1-4899-8e6a-26d12b85f19f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:28.303750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dbeb5-47d1-4899-8e6a-26d12b85f19f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:28.304162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dbeb5-47d1-4899-8e6a-26d12b85f19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:28.326575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0d6dbeb5-47d1-4899-8e6a-26d12b85f19f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:28.327940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0d6dbeb5-47d1-4899-8e6a-26d12b85f19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:28.400198 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1114/2231] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:28.410920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c6f7c-8fe6-480f-aacd-73d32fd3895b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:28.413055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f59c6f7c-8fe6-480f-aacd-73d32fd3895b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:28.481185 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c6f7c-8fe6-480f-aacd-73d32fd3895b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:28.485837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1ddc4-5e1f-4ac4-950a-0be002d8d7b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:28.487274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1ddc4-5e1f-4ac4-950a-0be002d8d7b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:28.487567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1ddc4-5e1f-4ac4-950a-0be002d8d7b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:28.487851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1ddc4-5e1f-4ac4-950a-0be002d8d7b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:28.499257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1ddc4-5e1f-4ac4-950a-0be002d8d7b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:28.500744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1ddc4-5e1f-4ac4-950a-0be002d8d7b9 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:28.512344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1118/2232] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:06:33.814618 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1dcd0-bec1-479d-bfab-018888dcecd2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:33.815206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1dcd0-bec1-479d-bfab-018888dcecd2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:33.815206 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1dcd0-bec1-479d-bfab-018888dcecd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:33.851688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-95a1dcd0-bec1-479d-bfab-018888dcecd2 None None] MFA Rules not processed for user `fa28f0e0b3b0488faf7db7bf7dc3854b`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:33.853148 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-95a1dcd0-bec1-479d-bfab-018888dcecd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:33.931487 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1115/2233] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:33.936145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1ddc4-5e1f-4ac4-950a-0be002d8d7b9 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:33.944415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:33.948943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1ddc4-5e1f-4ac4-950a-0be002d8d7b9 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:33.956322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333da61b-9d14-419a-b4c0-611d1988df3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:33.957328 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333da61b-9d14-419a-b4c0-611d1988df3c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:33.957744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333da61b-9d14-419a-b4c0-611d1988df3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:33.958301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333da61b-9d14-419a-b4c0-611d1988df3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:33.959539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333da61b-9d14-419a-b4c0-611d1988df3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:33.961900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333da61b-9d14-419a-b4c0-611d1988df3c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:33.994970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1119/2234] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:06:33 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:06:33.999192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1dcd0-bec1-479d-bfab-018888dcecd2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:34.014541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1dcd0-bec1-479d-bfab-018888dcecd2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:34.092845 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddef34c-e812-4dde-adc5-a6b9c0f86273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:34.093599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddef34c-e812-4dde-adc5-a6b9c0f86273 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:34.093914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddef34c-e812-4dde-adc5-a6b9c0f86273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:34.094187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddef34c-e812-4dde-adc5-a6b9c0f86273 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:34.094903 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddef34c-e812-4dde-adc5-a6b9c0f86273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:34.097018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddef34c-e812-4dde-adc5-a6b9c0f86273 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:34.106432 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:34.129735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1116/2235] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:06:33 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:06:34.133925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333da61b-9d14-419a-b4c0-611d1988df3c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:34.145998 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333da61b-9d14-419a-b4c0-611d1988df3c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:34.151013 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546592e1-8499-4a13-a07c-c80832a4c952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:34.151914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546592e1-8499-4a13-a07c-c80832a4c952 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:34.152253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546592e1-8499-4a13-a07c-c80832a4c952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:34.152555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546592e1-8499-4a13-a07c-c80832a4c952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:34.156481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546592e1-8499-4a13-a07c-c80832a4c952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:34.158444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546592e1-8499-4a13-a07c-c80832a4c952 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:34.176597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1120/2236] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:06:34.182384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddef34c-e812-4dde-adc5-a6b9c0f86273 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:34.192653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddef34c-e812-4dde-adc5-a6b9c0f86273 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:34.197783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6962c-170c-4ccc-b881-c1a23a9d82ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:34.198699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6962c-170c-4ccc-b881-c1a23a9d82ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:34.198977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6962c-170c-4ccc-b881-c1a23a9d82ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:34.199249 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6962c-170c-4ccc-b881-c1a23a9d82ab admin admin] PATH_INFO: `/v3/projects/1f91023a8d9e47c0b8eb1e8288abd7a3/users/8572f7d0b16f4c20abb90f5af9814609/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:34.224603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6962c-170c-4ccc-b881-c1a23a9d82ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f91023a8d9e47c0b8eb1e8288abd7a3, user_id=8572f7d0b16f4c20abb90f5af9814609, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:34.228179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6962c-170c-4ccc-b881-c1a23a9d82ab admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:34.256108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1117/2237] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:06:34 2022] PUT /identity/v3/projects/1f91023a8d9e47c0b8eb1e8288abd7a3/users/8572f7d0b16f4c20abb90f5af9814609/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:06:34.262591 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546592e1-8499-4a13-a07c-c80832a4c952 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:34.275419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546592e1-8499-4a13-a07c-c80832a4c952 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:34.365605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2b938-74fb-4038-8216-c47f67c77e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:34.366362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2b938-74fb-4038-8216-c47f67c77e58 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:34.366754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2b938-74fb-4038-8216-c47f67c77e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:34.366916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2b938-74fb-4038-8216-c47f67c77e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:34.367649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2b938-74fb-4038-8216-c47f67c77e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:34.369316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2b938-74fb-4038-8216-c47f67c77e58 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:34.372668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6962c-170c-4ccc-b881-c1a23a9d82ab admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:34.373320 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9da6962c-170c-4ccc-b881-c1a23a9d82ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:34.386376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1121/2238] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:06:34.392499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2b938-74fb-4038-8216-c47f67c77e58 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:34.411776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2b938-74fb-4038-8216-c47f67c77e58 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:34.417438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d42dbc-02c1-49ab-ad1b-278dc250ac7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:34.419546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d42dbc-02c1-49ab-ad1b-278dc250ac7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:34.419889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d42dbc-02c1-49ab-ad1b-278dc250ac7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:34.420323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d42dbc-02c1-49ab-ad1b-278dc250ac7a admin admin] PATH_INFO: `/v3/projects/1f91023a8d9e47c0b8eb1e8288abd7a3/users/8572f7d0b16f4c20abb90f5af9814609/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:34.423112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d42dbc-02c1-49ab-ad1b-278dc250ac7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f91023a8d9e47c0b8eb1e8288abd7a3, user_id=8572f7d0b16f4c20abb90f5af9814609, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:34.427138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d42dbc-02c1-49ab-ad1b-278dc250ac7a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:34.452674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1122/2239] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:06:34 2022] PUT /identity/v3/projects/1f91023a8d9e47c0b8eb1e8288abd7a3/users/8572f7d0b16f4c20abb90f5af9814609/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:06:34.530396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6962c-170c-4ccc-b881-c1a23a9d82ab admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:34.639380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7966743-c6ea-426b-b392-eb9fd8435584 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57dccfe9f5f4bc397f4c470fbf047dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:34.640468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7966743-c6ea-426b-b392-eb9fd8435584 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:34.640468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7966743-c6ea-426b-b392-eb9fd8435584 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:34.640942 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7966743-c6ea-426b-b392-eb9fd8435584 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:34.643445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f7966743-c6ea-426b-b392-eb9fd8435584 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:34.667277 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7966743-c6ea-426b-b392-eb9fd8435584 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:34.668844 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7966743-c6ea-426b-b392-eb9fd8435584 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:34.686869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1118/2240] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:06:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:06:37.959554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195ebc2-a09e-4847-8bc7-9d4ee370531a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:37.960670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195ebc2-a09e-4847-8bc7-9d4ee370531a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:37.961428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195ebc2-a09e-4847-8bc7-9d4ee370531a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:37.963531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1123/2241] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 08:06:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:06:37.972171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958d265-9963-4b69-b4ef-6ad4cb3d5038 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:37.972756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958d265-9963-4b69-b4ef-6ad4cb3d5038 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:37.972971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958d265-9963-4b69-b4ef-6ad4cb3d5038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:38.057321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c958d265-9963-4b69-b4ef-6ad4cb3d5038 None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:38.058565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c958d265-9963-4b69-b4ef-6ad4cb3d5038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:38.092535 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:38.211001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1119/2242] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 08:06:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:38.234842 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195ebc2-a09e-4847-8bc7-9d4ee370531a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:38.237500 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8195ebc2-a09e-4847-8bc7-9d4ee370531a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:38.357111 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195ebc2-a09e-4847-8bc7-9d4ee370531a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:38.361801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718dd93-feb9-4e38-a349-2dc553c020bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a122422a45446c9a63d67b4f931fa47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:38.362556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718dd93-feb9-4e38-a349-2dc553c020bd service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:38.362845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718dd93-feb9-4e38-a349-2dc553c020bd service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:38.363119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718dd93-feb9-4e38-a349-2dc553c020bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:38.375839 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718dd93-feb9-4e38-a349-2dc553c020bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:38.378945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718dd93-feb9-4e38-a349-2dc553c020bd service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:38.397657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1124/2243] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:06:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:06:38.510056 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e276ce-4378-4f2c-84c4-c7b2420c67c0 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:38.510371 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e276ce-4378-4f2c-84c4-c7b2420c67c0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:38.510678 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e276ce-4378-4f2c-84c4-c7b2420c67c0 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:38.512915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1120/2244] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 08:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:06:38.518739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f24ce-1d8a-4d67-8432-2f13a821037c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:38.519348 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f24ce-1d8a-4d67-8432-2f13a821037c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:38.519935 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f24ce-1d8a-4d67-8432-2f13a821037c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:38.549320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b62f24ce-1d8a-4d67-8432-2f13a821037c None None] MFA Rules not processed for user `429e173890b84ee289abc9793794ddb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:38.551328 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b62f24ce-1d8a-4d67-8432-2f13a821037c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:38.666314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1125/2245] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 08:06:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:38.670892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e276ce-4378-4f2c-84c4-c7b2420c67c0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:38.690721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e276ce-4378-4f2c-84c4-c7b2420c67c0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:38.705391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c13772-cd46-48f4-8c62-21f0d37291fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:38.707161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c13772-cd46-48f4-8c62-21f0d37291fc service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:38.707732 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c13772-cd46-48f4-8c62-21f0d37291fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:38.708232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c13772-cd46-48f4-8c62-21f0d37291fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:38.744584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c13772-cd46-48f4-8c62-21f0d37291fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:38.746848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c13772-cd46-48f4-8c62-21f0d37291fc service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:38.776945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1121/2246] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:06:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:06:38.987732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcac7a7-6462-4c0a-9e5e-705a76503b06 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:38.988269 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcac7a7-6462-4c0a-9e5e-705a76503b06 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:38.988269 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcac7a7-6462-4c0a-9e5e-705a76503b06 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:38.990002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1126/2247] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 08:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:06:39.007919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c684a-f1dc-4bd2-9366-ff7add8707ce None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:39.008781 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c684a-f1dc-4bd2-9366-ff7add8707ce None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:39.009237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c684a-f1dc-4bd2-9366-ff7add8707ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:39.060989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9e5c684a-f1dc-4bd2-9366-ff7add8707ce None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:39.062714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9e5c684a-f1dc-4bd2-9366-ff7add8707ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:39.080018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1122/2248] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 08:06:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:39.094569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcac7a7-6462-4c0a-9e5e-705a76503b06 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:39.116814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcac7a7-6462-4c0a-9e5e-705a76503b06 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:39.129936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d3ba7-49ad-4c81-9ed9-0d744bdcf213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:39.130799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d3ba7-49ad-4c81-9ed9-0d744bdcf213 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:39.131796 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d3ba7-49ad-4c81-9ed9-0d744bdcf213 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:39.132345 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d3ba7-49ad-4c81-9ed9-0d744bdcf213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:39.143141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:39.158921 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d3ba7-49ad-4c81-9ed9-0d744bdcf213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:39.168002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d3ba7-49ad-4c81-9ed9-0d744bdcf213 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:39.186074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1127/2249] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:06:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:06:41.896863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18d5b0-1843-4f15-b7b7-ce2ace9a80cf None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:41.897826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18d5b0-1843-4f15-b7b7-ce2ace9a80cf None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:41.897826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18d5b0-1843-4f15-b7b7-ce2ace9a80cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:41.950718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2a18d5b0-1843-4f15-b7b7-ce2ace9a80cf None None] MFA Rules not processed for user `8572f7d0b16f4c20abb90f5af9814609`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:41.953360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2a18d5b0-1843-4f15-b7b7-ce2ace9a80cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:42.085390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1123/2250] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:42.100944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d3ba7-49ad-4c81-9ed9-0d744bdcf213 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:42.101400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-545d3ba7-49ad-4c81-9ed9-0d744bdcf213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:42.155257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d3ba7-49ad-4c81-9ed9-0d744bdcf213 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:42.165798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97579c8b-b7ad-4d5d-b00e-c441265af90d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:42.167243 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97579c8b-b7ad-4d5d-b00e-c441265af90d service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:42.167529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97579c8b-b7ad-4d5d-b00e-c441265af90d service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:42.168290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97579c8b-b7ad-4d5d-b00e-c441265af90d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:42.264724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97579c8b-b7ad-4d5d-b00e-c441265af90d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:42.266594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97579c8b-b7ad-4d5d-b00e-c441265af90d service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:42.295086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1128/2251] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:06:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:06:48.021348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee02eb4-698b-4900-a9d3-b78040620d65 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:48.022444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee02eb4-698b-4900-a9d3-b78040620d65 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:48.022444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee02eb4-698b-4900-a9d3-b78040620d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:48.157881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3ee02eb4-698b-4900-a9d3-b78040620d65 None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:48.159908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3ee02eb4-698b-4900-a9d3-b78040620d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:48.235565 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:48.289966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1124/2252] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:06:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:06:48.307214 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97579c8b-b7ad-4d5d-b00e-c441265af90d service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:06:48.310363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-97579c8b-b7ad-4d5d-b00e-c441265af90d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:48.476257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97579c8b-b7ad-4d5d-b00e-c441265af90d service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:06:48.483350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f23c7-7edb-4961-95c1-7225d4c65363 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:06:48.484117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f23c7-7edb-4961-95c1-7225d4c65363 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:48.484497 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f23c7-7edb-4961-95c1-7225d4c65363 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:48.484898 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f23c7-7edb-4961-95c1-7225d4c65363 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:48.498693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f23c7-7edb-4961-95c1-7225d4c65363 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:06:48.503167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f23c7-7edb-4961-95c1-7225d4c65363 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:06:48.529512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1129/2253] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:06:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 08:06:58.280877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:58.516924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=72 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:06:59.546469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209466d-57f1-45c0-8830-48f1d3414f8a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:06:59.547316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209466d-57f1-45c0-8830-48f1d3414f8a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:06:59.547316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209466d-57f1-45c0-8830-48f1d3414f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:06:59.568405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5209466d-57f1-45c0-8830-48f1d3414f8a None None] MFA Rules not processed for user `34b2b4070d344d7ca07a99985678467a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:06:59.569969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5209466d-57f1-45c0-8830-48f1d3414f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:06:59.635790 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1125/2254] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:06:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:07:05.587550 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1ab03-02df-42fd-9719-0fc4206fd9c1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:07:05.588153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1ab03-02df-42fd-9719-0fc4206fd9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:07:05.588153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1ab03-02df-42fd-9719-0fc4206fd9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:07:05.620429 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-21c1ab03-02df-42fd-9719-0fc4206fd9c1 None None] MFA Rules not processed for user `fa28f0e0b3b0488faf7db7bf7dc3854b`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:07:05.621700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-21c1ab03-02df-42fd-9719-0fc4206fd9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:07:05.629207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1130/2255] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:07:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:07:09.627797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=25 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:07:14.973458 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1536e-a1aa-4b43-a399-c73438b922b8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:07:14.974892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1536e-a1aa-4b43-a399-c73438b922b8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:07:14.974892 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1536e-a1aa-4b43-a399-c73438b922b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:07:15.024702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e5e1536e-a1aa-4b43-a399-c73438b922b8 None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:07:15.026288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e5e1536e-a1aa-4b43-a399-c73438b922b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:07:15.045760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1126/2256] 10.222.0.43 () {58 vars in 967 bytes} [Mon Jul 4 08:07:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:07:15.083097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1ab03-02df-42fd-9719-0fc4206fd9c1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:07:15.116696 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1ab03-02df-42fd-9719-0fc4206fd9c1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:07:15.130134 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c695b-31e8-46a0-9548-2cf6ed252be2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:07:15.132564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c695b-31e8-46a0-9548-2cf6ed252be2 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:07:15.133573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c695b-31e8-46a0-9548-2cf6ed252be2 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:07:15.134378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c695b-31e8-46a0-9548-2cf6ed252be2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:07:15.175551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c695b-31e8-46a0-9548-2cf6ed252be2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:07:15.180766 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c695b-31e8-46a0-9548-2cf6ed252be2 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:07:15.222566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1131/2257] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:07:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 08:07:23.664538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1536e-a1aa-4b43-a399-c73438b922b8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:07:23.666074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e5e1536e-a1aa-4b43-a399-c73438b922b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:07:23.686683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1536e-a1aa-4b43-a399-c73438b922b8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:07:23.691780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31239be3-284b-4cb1-bca7-d76820256bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:07:23.692899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31239be3-284b-4cb1-bca7-d76820256bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:07:23.693445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31239be3-284b-4cb1-bca7-d76820256bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:07:23.693969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31239be3-284b-4cb1-bca7-d76820256bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:07:23.696296 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-31239be3-284b-4cb1-bca7-d76820256bb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:07:23.847034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31239be3-284b-4cb1-bca7-d76820256bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:07:23.857681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31239be3-284b-4cb1-bca7-d76820256bb4 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:07:23.878876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1127/2258] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:07:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:07:24.391150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc9631-3f81-4e72-87da-9ab6791c5fb6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:07:24.393102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc9631-3f81-4e72-87da-9ab6791c5fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:07:24.393940 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc9631-3f81-4e72-87da-9ab6791c5fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:07:24.450283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-15bc9631-3f81-4e72-87da-9ab6791c5fb6 None None] MFA Rules not processed for user `9e3f73e469f34b66bfbe99c3d79ac129`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:07:24.452785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-15bc9631-3f81-4e72-87da-9ab6791c5fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:07:24.573693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1132/2259] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:07:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:07:33.867873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=33 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:07:34.557842 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=34 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:07:45.237653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31239be3-284b-4cb1-bca7-d76820256bb4 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:07:45.246409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31239be3-284b-4cb1-bca7-d76820256bb4 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:07:45.250102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e11d30-b10a-42cc-8662-ba29b3301588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:07:45.250738 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e11d30-b10a-42cc-8662-ba29b3301588 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:07:45.251013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e11d30-b10a-42cc-8662-ba29b3301588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:07:45.251285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e11d30-b10a-42cc-8662-ba29b3301588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:07:45.261310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e11d30-b10a-42cc-8662-ba29b3301588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:07:45.262768 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e11d30-b10a-42cc-8662-ba29b3301588 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:07:45.274392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1128/2260] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:07:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:07:55.268582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:07:56.593670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba75d-6abc-4cb2-bb2f-4f6d780eb677 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:07:56.594267 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba75d-6abc-4cb2-bb2f-4f6d780eb677 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:07:56.594267 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba75d-6abc-4cb2-bb2f-4f6d780eb677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:07:56.613152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c4cba75d-6abc-4cb2-bb2f-4f6d780eb677 None None] MFA Rules not processed for user `126ba607541e4c4895b47b4b7602ce01`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:07:56.614442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c4cba75d-6abc-4cb2-bb2f-4f6d780eb677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:07:56.678971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1133/2261] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:07:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:08:06.671633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=25 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:08:20.102967 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad823d-e33c-4233-9633-7f15f8e99f04 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:08:20.104513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad823d-e33c-4233-9633-7f15f8e99f04 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:08:20.105277 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad823d-e33c-4233-9633-7f15f8e99f04 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:08:20.115531 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1129/2262] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 08:08:20 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:08:20.128513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e041e2-cebc-4fd0-b431-68cf49584b6c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:08:20.129123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e041e2-cebc-4fd0-b431-68cf49584b6c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:08:20.129731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e041e2-cebc-4fd0-b431-68cf49584b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:08:20.188603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad823d-e33c-4233-9633-7f15f8e99f04 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:08:20.190571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-60ad823d-e33c-4233-9633-7f15f8e99f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:20.204366 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-33e041e2-cebc-4fd0-b431-68cf49584b6c None None] MFA Rules not processed for user `f57dccfe9f5f4bc397f4c470fbf047dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:08:20.205899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-33e041e2-cebc-4fd0-b431-68cf49584b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:20.218805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1134/2263] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 08:08:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:08:20.234177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e041e2-cebc-4fd0-b431-68cf49584b6c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:08:20.252921 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e041e2-cebc-4fd0-b431-68cf49584b6c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:08:20.259037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94b298-f656-45a5-b168-e4723930c933 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57dccfe9f5f4bc397f4c470fbf047dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:08:20.260106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94b298-f656-45a5-b168-e4723930c933 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:08:20.260391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94b298-f656-45a5-b168-e4723930c933 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:08:20.260672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94b298-f656-45a5-b168-e4723930c933 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:08:20.270029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94b298-f656-45a5-b168-e4723930c933 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:08:20.271706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94b298-f656-45a5-b168-e4723930c933 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:08:20.283999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1135/2264] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:08:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:08:20.296226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad823d-e33c-4233-9633-7f15f8e99f04 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:08:20.303609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036ad4c-6681-480b-b671-3e87ed845c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:08:20.304267 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036ad4c-6681-480b-b671-3e87ed845c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:08:20.304561 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036ad4c-6681-480b-b671-3e87ed845c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:08:20.304854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036ad4c-6681-480b-b671-3e87ed845c69 admin admin] PATH_INFO: `/v3/users/34b2b4070d344d7ca07a99985678467a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:08:20.306214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036ad4c-6681-480b-b671-3e87ed845c69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b2b4070d344d7ca07a99985678467a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:08:20.309452 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036ad4c-6681-480b-b671-3e87ed845c69 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:08:20.362360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2036ad4c-6681-480b-b671-3e87ed845c69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b2b4070d344d7ca07a99985678467a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:08:20.363134 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2036ad4c-6681-480b-b671-3e87ed845c69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b2b4070d344d7ca07a99985678467a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:08:20.363812 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2036ad4c-6681-480b-b671-3e87ed845c69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b2b4070d344d7ca07a99985678467a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:08:20.372460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2036ad4c-6681-480b-b671-3e87ed845c69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b2b4070d344d7ca07a99985678467a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:08:20.387049 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2036ad4c-6681-480b-b671-3e87ed845c69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b2b4070d344d7ca07a99985678467a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:08:20.413154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1130/2265] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:08:20 2022] DELETE /identity/v3/users/34b2b4070d344d7ca07a99985678467a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:08:20.624556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94b298-f656-45a5-b168-e4723930c933 service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:08:20.637509 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94b298-f656-45a5-b168-e4723930c933 service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:08:20.699382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:08:20.700263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:08:20.700537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:08:20.700808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] PATH_INFO: `/v3/projects/6de1b2d9c55f4af7b6793ac472a6d408` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:08:20.702095 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6de1b2d9c55f4af7b6793ac472a6d408)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:08:20.704044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:08:20.759837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6de1b2d9c55f4af7b6793ac472a6d408', 'request_id': 'req-acad1edd-ed01-4956-a0b0-187adc18732b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:08:20.765781 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6de1b2d9c55f4af7b6793ac472a6d408', 'request_id': 'req-acad1edd-ed01-4956-a0b0-187adc18732b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:08:20.774507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6de1b2d9c55f4af7b6793ac472a6d408', 'request_id': 'req-acad1edd-ed01-4956-a0b0-187adc18732b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:08:20.788997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] The token cache is being invalidate because project 6de1b2d9c55f4af7b6793ac472a6d408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:08:20.789295 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:08:20.790449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:08:20.792993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1136/2266] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:08:20 2022] DELETE /identity/v3/projects/6de1b2d9c55f4af7b6793ac472a6d408 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:08:27.112578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca2051-57d4-4fca-bd0b-8efcf4bdc876 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:08:27.113032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca2051-57d4-4fca-bd0b-8efcf4bdc876 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:08:27.113119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca2051-57d4-4fca-bd0b-8efcf4bdc876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:08:27.133577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e4ca2051-57d4-4fca-bd0b-8efcf4bdc876 None None] MFA Rules not processed for user `1501eb69e0a447ce83baa5bdee770d1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:08:27.134888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e4ca2051-57d4-4fca-bd0b-8efcf4bdc876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:27.195136 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1131/2267] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:08:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:08:27.865530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:08:27.867240 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:27.932920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad1edd-ed01-4956-a0b0-187adc18732b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:08:27.936945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9d56b-fc91-4e36-b920-0928f84a3b89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:08:27.937599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9d56b-fc91-4e36-b920-0928f84a3b89 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:08:27.937923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9d56b-fc91-4e36-b920-0928f84a3b89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:08:27.938194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9d56b-fc91-4e36-b920-0928f84a3b89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:08:27.940282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3ae9d56b-fc91-4e36-b920-0928f84a3b89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:27.951876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:08:28.019541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9d56b-fc91-4e36-b920-0928f84a3b89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:08:28.020978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9d56b-fc91-4e36-b920-0928f84a3b89 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:08:28.037842 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1137/2268] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:08:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:08:32.956002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca2051-57d4-4fca-bd0b-8efcf4bdc876 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:08:32.957534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e4ca2051-57d4-4fca-bd0b-8efcf4bdc876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:32.970189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca2051-57d4-4fca-bd0b-8efcf4bdc876 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:08:32.974431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4c79f-1223-4b9d-b265-eaab68cfb41b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:08:32.975017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4c79f-1223-4b9d-b265-eaab68cfb41b service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:08:32.975304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4c79f-1223-4b9d-b265-eaab68cfb41b service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:08:32.975587 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4c79f-1223-4b9d-b265-eaab68cfb41b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:08:32.977447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-22d4c79f-1223-4b9d-b265-eaab68cfb41b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:32.991647 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4c79f-1223-4b9d-b265-eaab68cfb41b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:08:32.993098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4c79f-1223-4b9d-b265-eaab68cfb41b service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:08:33.010641 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1132/2269] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:08:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:08:38.028150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=28 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:08:43.001299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:08:46.667783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9d56b-fc91-4e36-b920-0928f84a3b89 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:08:46.669338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3ae9d56b-fc91-4e36-b920-0928f84a3b89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:46.756732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9d56b-fc91-4e36-b920-0928f84a3b89 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:08:46.761055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe572d3d-7eaf-4684-8ea2-4aaf2d1a1234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:08:46.761850 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe572d3d-7eaf-4684-8ea2-4aaf2d1a1234 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:08:46.762041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe572d3d-7eaf-4684-8ea2-4aaf2d1a1234 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:08:46.762311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe572d3d-7eaf-4684-8ea2-4aaf2d1a1234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:08:46.764241 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fe572d3d-7eaf-4684-8ea2-4aaf2d1a1234 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:46.826374 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe572d3d-7eaf-4684-8ea2-4aaf2d1a1234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:08:46.827830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe572d3d-7eaf-4684-8ea2-4aaf2d1a1234 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:08:46.839626 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1138/2270] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:08:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:08:54.412021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909c818-eb63-4c0f-a77a-563e2f5fb217 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:08:54.412582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909c818-eb63-4c0f-a77a-563e2f5fb217 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:08:54.412582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909c818-eb63-4c0f-a77a-563e2f5fb217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:08:54.438726 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b909c818-eb63-4c0f-a77a-563e2f5fb217 None None] MFA Rules not processed for user `18b1072539af49729dff7ee9ea445c77`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:08:54.439966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b909c818-eb63-4c0f-a77a-563e2f5fb217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:54.447971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1133/2271] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:08:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:08:54.784112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe572d3d-7eaf-4684-8ea2-4aaf2d1a1234 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:08:54.801930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe572d3d-7eaf-4684-8ea2-4aaf2d1a1234 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:08:54.805648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01fd86-dcae-418d-89b6-4149123b9069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:08:54.806670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01fd86-dcae-418d-89b6-4149123b9069 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:08:54.806670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01fd86-dcae-418d-89b6-4149123b9069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:08:54.806955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01fd86-dcae-418d-89b6-4149123b9069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:08:54.808888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8d01fd86-dcae-418d-89b6-4149123b9069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:54.819931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01fd86-dcae-418d-89b6-4149123b9069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:08:54.821386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01fd86-dcae-418d-89b6-4149123b9069 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:08:54.832313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1139/2272] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:08:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:08:55.185093 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909c818-eb63-4c0f-a77a-563e2f5fb217 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:08:55.186594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b909c818-eb63-4c0f-a77a-563e2f5fb217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:55.271654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909c818-eb63-4c0f-a77a-563e2f5fb217 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:08:55.277194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcb3a1-2e94-4bc2-8df0-3368cd278421 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57dccfe9f5f4bc397f4c470fbf047dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:08:55.277912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcb3a1-2e94-4bc2-8df0-3368cd278421 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:08:55.278310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcb3a1-2e94-4bc2-8df0-3368cd278421 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:08:55.278471 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcb3a1-2e94-4bc2-8df0-3368cd278421 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:08:55.280239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e2fcb3a1-2e94-4bc2-8df0-3368cd278421 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:08:55.292499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcb3a1-2e94-4bc2-8df0-3368cd278421 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:08:55.294171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcb3a1-2e94-4bc2-8df0-3368cd278421 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:08:55.305322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1134/2273] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:08:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:09:02.744376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01fd86-dcae-418d-89b6-4149123b9069 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:09:02.745966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8d01fd86-dcae-418d-89b6-4149123b9069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:02.774652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01fd86-dcae-418d-89b6-4149123b9069 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:09:02.783350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dc3dc-a24c-4ed7-aecc-7b183b3acfa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:09:02.784008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dc3dc-a24c-4ed7-aecc-7b183b3acfa4 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:02.784290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dc3dc-a24c-4ed7-aecc-7b183b3acfa4 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:02.784560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dc3dc-a24c-4ed7-aecc-7b183b3acfa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:02.786563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c55dc3dc-a24c-4ed7-aecc-7b183b3acfa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:02.854210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dc3dc-a24c-4ed7-aecc-7b183b3acfa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:09:02.855894 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dc3dc-a24c-4ed7-aecc-7b183b3acfa4 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:09:02.866036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1140/2274] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:09:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:09:04.079141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcb3a1-2e94-4bc2-8df0-3368cd278421 service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:09:04.089331 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcb3a1-2e94-4bc2-8df0-3368cd278421 service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:09:04.094265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4cc07-a94c-45ce-877c-c2d4d6306991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:09:04.094907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4cc07-a94c-45ce-877c-c2d4d6306991 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:04.095195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4cc07-a94c-45ce-877c-c2d4d6306991 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:04.095467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4cc07-a94c-45ce-877c-c2d4d6306991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:04.097654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-97d4cc07-a94c-45ce-877c-c2d4d6306991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:04.108496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4cc07-a94c-45ce-877c-c2d4d6306991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:09:04.110055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4cc07-a94c-45ce-877c-c2d4d6306991 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:09:04.118828 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1135/2275] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:09:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:09:10.457031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e04a0-2c61-46d5-94ad-514bbcfb6524 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:10.457696 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e04a0-2c61-46d5-94ad-514bbcfb6524 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:10.457696 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e04a0-2c61-46d5-94ad-514bbcfb6524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:10.464255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:09:10.478300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1d9e04a0-2c61-46d5-94ad-514bbcfb6524 None None] MFA Rules not processed for user `9e3f73e469f34b66bfbe99c3d79ac129`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:09:10.479600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1d9e04a0-2c61-46d5-94ad-514bbcfb6524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:10.547757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1141/2276] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:09:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:09:14.114915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=41 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:09:17.697761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4cc07-a94c-45ce-877c-c2d4d6306991 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:09:17.713898 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4cc07-a94c-45ce-877c-c2d4d6306991 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:09:17.720469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62aeafc-e5d0-4c9d-a80c-03fd0fa816b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:09:17.721252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62aeafc-e5d0-4c9d-a80c-03fd0fa816b1 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:17.721598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62aeafc-e5d0-4c9d-a80c-03fd0fa816b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:17.721969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62aeafc-e5d0-4c9d-a80c-03fd0fa816b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:17.724497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a62aeafc-e5d0-4c9d-a80c-03fd0fa816b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:17.739580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62aeafc-e5d0-4c9d-a80c-03fd0fa816b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:09:17.741259 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62aeafc-e5d0-4c9d-a80c-03fd0fa816b1 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:09:17.755335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1136/2277] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:09:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:09:20.540595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=24 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:09:20.696038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e04a0-2c61-46d5-94ad-514bbcfb6524 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:09:20.711164 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e04a0-2c61-46d5-94ad-514bbcfb6524 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:09:20.741995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7739ed-d602-4fe7-b9f9-91dbd0b6f2ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:09:20.743163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7739ed-d602-4fe7-b9f9-91dbd0b6f2ff service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:20.743723 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7739ed-d602-4fe7-b9f9-91dbd0b6f2ff service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:20.744588 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7739ed-d602-4fe7-b9f9-91dbd0b6f2ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:20.748264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9d7739ed-d602-4fe7-b9f9-91dbd0b6f2ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:20.774986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7739ed-d602-4fe7-b9f9-91dbd0b6f2ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:09:20.778259 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7739ed-d602-4fe7-b9f9-91dbd0b6f2ff service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:09:20.800332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1142/2278] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:09:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:09:24.831543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a933bb-5b03-461c-afdc-8787b54fe981 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:24.832121 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a933bb-5b03-461c-afdc-8787b54fe981 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:24.832551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a933bb-5b03-461c-afdc-8787b54fe981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:24.864309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-63a933bb-5b03-461c-afdc-8787b54fe981 None None] MFA Rules not processed for user `126ba607541e4c4895b47b4b7602ce01`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:09:24.866497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-63a933bb-5b03-461c-afdc-8787b54fe981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:24.879804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1137/2279] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:09:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:09:24.951298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7739ed-d602-4fe7-b9f9-91dbd0b6f2ff service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:09:24.966065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7739ed-d602-4fe7-b9f9-91dbd0b6f2ff service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:09:24.972788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b77bd-54b7-46e9-880c-5e0a59f1c104 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:09:24.973465 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b77bd-54b7-46e9-880c-5e0a59f1c104 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:24.973779 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b77bd-54b7-46e9-880c-5e0a59f1c104 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:24.974062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b77bd-54b7-46e9-880c-5e0a59f1c104 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:24.976054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3b6b77bd-54b7-46e9-880c-5e0a59f1c104 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:24.988834 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b77bd-54b7-46e9-880c-5e0a59f1c104 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:09:24.990546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b77bd-54b7-46e9-880c-5e0a59f1c104 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:09:25.002064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1143/2280] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:09:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:09:34.099554 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a933bb-5b03-461c-afdc-8787b54fe981 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:09:34.115857 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a933bb-5b03-461c-afdc-8787b54fe981 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:09:34.123045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23105d-f54b-4738-a0b6-e5473211e293 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:09:34.123936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23105d-f54b-4738-a0b6-e5473211e293 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:34.124491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23105d-f54b-4738-a0b6-e5473211e293 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:34.124929 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23105d-f54b-4738-a0b6-e5473211e293 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:34.127227 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6c23105d-f54b-4738-a0b6-e5473211e293 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:34.142172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23105d-f54b-4738-a0b6-e5473211e293 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:09:34.143817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23105d-f54b-4738-a0b6-e5473211e293 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:09:34.165051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1138/2281] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:09:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:09:34.996590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=12 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:09:41.446432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3263b0a-784b-4d7d-937a-276cdae5b59b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:41.447123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3263b0a-784b-4d7d-937a-276cdae5b59b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:41.447123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3263b0a-784b-4d7d-937a-276cdae5b59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:41.469176 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e3263b0a-784b-4d7d-937a-276cdae5b59b None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:09:41.471795 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e3263b0a-784b-4d7d-937a-276cdae5b59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:41.533539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1144/2282] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:09:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:09:41.593609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23105d-f54b-4738-a0b6-e5473211e293 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:09:41.595225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6c23105d-f54b-4738-a0b6-e5473211e293 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:41.669975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23105d-f54b-4738-a0b6-e5473211e293 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:09:41.676039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8f795-75bb-4f0c-a865-a21390e4b31f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:09:41.676716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8f795-75bb-4f0c-a865-a21390e4b31f service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:41.676999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8f795-75bb-4f0c-a865-a21390e4b31f service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:41.677281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8f795-75bb-4f0c-a865-a21390e4b31f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:41.686957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8f795-75bb-4f0c-a865-a21390e4b31f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:09:41.688599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8f795-75bb-4f0c-a865-a21390e4b31f service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:09:41.698015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1139/2283] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:09:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 08:09:51.526073 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=25 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:09:51.694447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=47 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:09:56.431402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3263b0a-784b-4d7d-937a-276cdae5b59b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:09:56.441266 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3263b0a-784b-4d7d-937a-276cdae5b59b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:09:56.445221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743c7d3-83ca-470f-bfd6-49c77f7d37ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:09:56.445938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743c7d3-83ca-470f-bfd6-49c77f7d37ea service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:56.446218 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743c7d3-83ca-470f-bfd6-49c77f7d37ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:56.446491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743c7d3-83ca-470f-bfd6-49c77f7d37ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:56.448345 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c743c7d3-83ca-470f-bfd6-49c77f7d37ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:56.458699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743c7d3-83ca-470f-bfd6-49c77f7d37ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:09:56.460118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743c7d3-83ca-470f-bfd6-49c77f7d37ea service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:09:56.475902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1145/2284] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:09:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:09:56.592129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98c656-7f95-4b83-9315-42a76908597b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:56.592961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98c656-7f95-4b83-9315-42a76908597b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:56.592961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98c656-7f95-4b83-9315-42a76908597b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:56.594466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1140/2285] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 08:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 08:09:56.602221 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277005a7-4277-4798-b20d-f259e15f027b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:56.602508 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277005a7-4277-4798-b20d-f259e15f027b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:56.602786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277005a7-4277-4798-b20d-f259e15f027b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:56.622219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-277005a7-4277-4798-b20d-f259e15f027b None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:09:56.623457 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-277005a7-4277-4798-b20d-f259e15f027b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:56.633432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1146/2286] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 08:09:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:09:56.646797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98c656-7f95-4b83-9315-42a76908597b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:09:56.648998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eb98c656-7f95-4b83-9315-42a76908597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:56.741322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98c656-7f95-4b83-9315-42a76908597b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:09:56.745433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84853c2d-1195-4cdd-bc5c-45a865f17f25 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a122422a45446c9a63d67b4f931fa47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:09:56.746236 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84853c2d-1195-4cdd-bc5c-45a865f17f25 service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:56.746591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84853c2d-1195-4cdd-bc5c-45a865f17f25 service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:56.746935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84853c2d-1195-4cdd-bc5c-45a865f17f25 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:56.759431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84853c2d-1195-4cdd-bc5c-45a865f17f25 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:09:56.760889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84853c2d-1195-4cdd-bc5c-45a865f17f25 service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:09:56.772397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1141/2287] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:09:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:09:56.847362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cd4a1-e7be-449f-9131-252a98db43ce None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:56.847651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cd4a1-e7be-449f-9131-252a98db43ce None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:56.847934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cd4a1-e7be-449f-9131-252a98db43ce None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:56.849335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1147/2288] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 08:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:09:56.855146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff3d39-c9f7-4867-9a2f-7bcd415aff10 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:56.855453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff3d39-c9f7-4867-9a2f-7bcd415aff10 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:56.855729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff3d39-c9f7-4867-9a2f-7bcd415aff10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:56.876617 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-14ff3d39-c9f7-4867-9a2f-7bcd415aff10 None None] MFA Rules not processed for user `429e173890b84ee289abc9793794ddb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:09:56.878981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-14ff3d39-c9f7-4867-9a2f-7bcd415aff10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:56.890527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1142/2289] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 08:09:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:09:56.909372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cd4a1-e7be-449f-9131-252a98db43ce None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:09:56.924573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cd4a1-e7be-449f-9131-252a98db43ce None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:09:56.928504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c28958-bbdc-46e0-b67a-766935558c06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:09:56.929155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c28958-bbdc-46e0-b67a-766935558c06 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:56.929441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c28958-bbdc-46e0-b67a-766935558c06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:56.929751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c28958-bbdc-46e0-b67a-766935558c06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:56.937787 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c28958-bbdc-46e0-b67a-766935558c06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:09:56.939444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c28958-bbdc-46e0-b67a-766935558c06 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:09:56.949120 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1148/2290] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:09:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:09:57.041164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897ec91-6072-455e-b51d-627177f60fba None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:57.041458 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897ec91-6072-455e-b51d-627177f60fba None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:57.041769 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897ec91-6072-455e-b51d-627177f60fba None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:57.043195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1143/2291] 10.222.0.43 () {54 vars in 921 bytes} [Mon Jul 4 08:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 08:09:57.049592 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df5b66-4f90-42fa-b474-26a64bdb88d0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:57.049806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df5b66-4f90-42fa-b474-26a64bdb88d0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:57.050098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df5b66-4f90-42fa-b474-26a64bdb88d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:57.070111 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-55df5b66-4f90-42fa-b474-26a64bdb88d0 None None] MFA Rules not processed for user `15211e42ec9d4c1699d877f0b55ce918`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:09:57.071371 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-55df5b66-4f90-42fa-b474-26a64bdb88d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:09:57.082332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1149/2292] 10.222.0.43 () {58 vars in 1018 bytes} [Mon Jul 4 08:09:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:09:57.094336 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897ec91-6072-455e-b51d-627177f60fba None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:09:57.111045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897ec91-6072-455e-b51d-627177f60fba None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:09:57.114771 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0e2f3-2676-4adb-afa7-305cbf194106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:09:57.115440 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0e2f3-2676-4adb-afa7-305cbf194106 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:09:57.115739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0e2f3-2676-4adb-afa7-305cbf194106 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:09:57.115990 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0e2f3-2676-4adb-afa7-305cbf194106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:09:57.127097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0e2f3-2676-4adb-afa7-305cbf194106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:09:57.128879 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0e2f3-2676-4adb-afa7-305cbf194106 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:09:57.141465 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1144/2293] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:09:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:10:02.218305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aa97a-cdda-4dca-8a8b-f4ded3e47bf6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:02.218988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aa97a-cdda-4dca-8a8b-f4ded3e47bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:02.218988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aa97a-cdda-4dca-8a8b-f4ded3e47bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:02.240907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2b8aa97a-cdda-4dca-8a8b-f4ded3e47bf6 None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:10:02.242085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2b8aa97a-cdda-4dca-8a8b-f4ded3e47bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:02.250514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1150/2294] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:10:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:10:02.266667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0e2f3-2676-4adb-afa7-305cbf194106 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:02.282379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0e2f3-2676-4adb-afa7-305cbf194106 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:02.288613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d35ec4-c786-4352-b608-caf251cf684e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:02.289369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d35ec4-c786-4352-b608-caf251cf684e service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:02.289759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d35ec4-c786-4352-b608-caf251cf684e service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:02.290109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d35ec4-c786-4352-b608-caf251cf684e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:02.301200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d35ec4-c786-4352-b608-caf251cf684e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:02.302883 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d35ec4-c786-4352-b608-caf251cf684e service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:02.322202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1145/2295] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:10:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 08:10:12.234027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=21 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:10:12.308827 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=50 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:10:14.613706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8aa97a-cdda-4dca-8a8b-f4ded3e47bf6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:14.629372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8aa97a-cdda-4dca-8a8b-f4ded3e47bf6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:14.643510 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38b44e-a9cd-4150-81c9-288b7c9b0f17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:14.644207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38b44e-a9cd-4150-81c9-288b7c9b0f17 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:14.644518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38b44e-a9cd-4150-81c9-288b7c9b0f17 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:14.644767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38b44e-a9cd-4150-81c9-288b7c9b0f17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:14.646748 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-af38b44e-a9cd-4150-81c9-288b7c9b0f17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:14.658562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af38b44e-a9cd-4150-81c9-288b7c9b0f17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:14.660214 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af38b44e-a9cd-4150-81c9-288b7c9b0f17 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:14.670718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1151/2296] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:10:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:10:21.877116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff81c7-7d2c-456c-8cfc-9eb8059514b6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:21.879838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff81c7-7d2c-456c-8cfc-9eb8059514b6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:21.880805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff81c7-7d2c-456c-8cfc-9eb8059514b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:21.904993 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e6ff81c7-7d2c-456c-8cfc-9eb8059514b6 None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:10:21.906510 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e6ff81c7-7d2c-456c-8cfc-9eb8059514b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:21.916652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1146/2297] 10.222.0.43 () {58 vars in 967 bytes} [Mon Jul 4 08:10:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:10:21.932149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38b44e-a9cd-4150-81c9-288b7c9b0f17 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:21.944497 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38b44e-a9cd-4150-81c9-288b7c9b0f17 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:21.951402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a223f-c19d-474e-aaa4-3e383a0e07d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:21.952187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a223f-c19d-474e-aaa4-3e383a0e07d9 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:21.952541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a223f-c19d-474e-aaa4-3e383a0e07d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:21.952882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a223f-c19d-474e-aaa4-3e383a0e07d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:21.964223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a223f-c19d-474e-aaa4-3e383a0e07d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:21.966133 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a223f-c19d-474e-aaa4-3e383a0e07d9 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:21.976830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1152/2298] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:10:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 08:10:23.064687 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f0747-b985-43db-b012-2aeec0f5262f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:23.065233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f0747-b985-43db-b012-2aeec0f5262f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:23.065233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f0747-b985-43db-b012-2aeec0f5262f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:23.084317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-057f0747-b985-43db-b012-2aeec0f5262f None None] MFA Rules not processed for user `fa28f0e0b3b0488faf7db7bf7dc3854b`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:10:23.085717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-057f0747-b985-43db-b012-2aeec0f5262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:23.154140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1147/2299] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:10:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:10:31.971957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=12 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:10:33.146077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=28 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:10:41.951719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a223f-c19d-474e-aaa4-3e383a0e07d9 service swift] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:41.961978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a223f-c19d-474e-aaa4-3e383a0e07d9 service swift] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:41.966174 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da98463-1da8-4f3f-88ae-5ff0d019206c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:41.966746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da98463-1da8-4f3f-88ae-5ff0d019206c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:41.967026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da98463-1da8-4f3f-88ae-5ff0d019206c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:41.967301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da98463-1da8-4f3f-88ae-5ff0d019206c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:41.969458 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3da98463-1da8-4f3f-88ae-5ff0d019206c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:42.042067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da98463-1da8-4f3f-88ae-5ff0d019206c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:42.043533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da98463-1da8-4f3f-88ae-5ff0d019206c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:42.053769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1153/2300] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:10:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:10:47.796735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f0747-b985-43db-b012-2aeec0f5262f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:47.847319 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f0747-b985-43db-b012-2aeec0f5262f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:47.875087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c0bb9-1a9d-46b5-9504-3ba1ed130a3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57dccfe9f5f4bc397f4c470fbf047dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:47.875798 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c0bb9-1a9d-46b5-9504-3ba1ed130a3b service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:47.876083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c0bb9-1a9d-46b5-9504-3ba1ed130a3b service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:47.876361 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c0bb9-1a9d-46b5-9504-3ba1ed130a3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:47.886863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7a2c0bb9-1a9d-46b5-9504-3ba1ed130a3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:47.937865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c0bb9-1a9d-46b5-9504-3ba1ed130a3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:47.947377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da98463-1da8-4f3f-88ae-5ff0d019206c service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:47.948372 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c0bb9-1a9d-46b5-9504-3ba1ed130a3b service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:47.950088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3da98463-1da8-4f3f-88ae-5ff0d019206c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:47.974203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da98463-1da8-4f3f-88ae-5ff0d019206c service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:47.974961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1148/2301] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:10:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:10:47.996353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab396afe-36f8-4155-9c95-a7941c6e3546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:47.997647 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab396afe-36f8-4155-9c95-a7941c6e3546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:47.998138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab396afe-36f8-4155-9c95-a7941c6e3546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:47.999325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab396afe-36f8-4155-9c95-a7941c6e3546 admin admin] PATH_INFO: `/v3/users/9e3f73e469f34b66bfbe99c3d79ac129` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:48.000935 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab396afe-36f8-4155-9c95-a7941c6e3546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e3f73e469f34b66bfbe99c3d79ac129)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:48.003594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab396afe-36f8-4155-9c95-a7941c6e3546 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:48.085050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab396afe-36f8-4155-9c95-a7941c6e3546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e3f73e469f34b66bfbe99c3d79ac129'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:48.098053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab396afe-36f8-4155-9c95-a7941c6e3546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e3f73e469f34b66bfbe99c3d79ac129'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:48.125562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab396afe-36f8-4155-9c95-a7941c6e3546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e3f73e469f34b66bfbe99c3d79ac129'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:48.130415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab396afe-36f8-4155-9c95-a7941c6e3546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e3f73e469f34b66bfbe99c3d79ac129'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:48.162478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab396afe-36f8-4155-9c95-a7941c6e3546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e3f73e469f34b66bfbe99c3d79ac129'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:48.177488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1154/2302] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:10:47 2022] DELETE /identity/v3/users/9e3f73e469f34b66bfbe99c3d79ac129 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:10:49.277585 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c0bb9-1a9d-46b5-9504-3ba1ed130a3b service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:49.289467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c0bb9-1a9d-46b5-9504-3ba1ed130a3b service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:49.364086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844247f1-a040-4583-9855-38db0182c2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:49.364727 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844247f1-a040-4583-9855-38db0182c2dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:49.365025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844247f1-a040-4583-9855-38db0182c2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:49.365288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844247f1-a040-4583-9855-38db0182c2dc admin admin] PATH_INFO: `/v3/projects/ecd5bc804a3b4f14b864fb4e0f8d1b98` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:49.367192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844247f1-a040-4583-9855-38db0182c2dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecd5bc804a3b4f14b864fb4e0f8d1b98)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:49.369360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844247f1-a040-4583-9855-38db0182c2dc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:49.425145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-844247f1-a040-4583-9855-38db0182c2dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd5bc804a3b4f14b864fb4e0f8d1b98', 'request_id': 'req-844247f1-a040-4583-9855-38db0182c2dc'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:49.433073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-844247f1-a040-4583-9855-38db0182c2dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd5bc804a3b4f14b864fb4e0f8d1b98', 'request_id': 'req-844247f1-a040-4583-9855-38db0182c2dc'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:49.440580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-844247f1-a040-4583-9855-38db0182c2dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd5bc804a3b4f14b864fb4e0f8d1b98', 'request_id': 'req-844247f1-a040-4583-9855-38db0182c2dc'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:49.457786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-844247f1-a040-4583-9855-38db0182c2dc admin admin] The token cache is being invalidate because project ecd5bc804a3b4f14b864fb4e0f8d1b98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:10:49.458166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-844247f1-a040-4583-9855-38db0182c2dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:49.458836 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-844247f1-a040-4583-9855-38db0182c2dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:49.469032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1149/2303] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:10:49 2022] DELETE /identity/v3/projects/ecd5bc804a3b4f14b864fb4e0f8d1b98 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:10:49.472489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbe4c5-7584-466b-8078-1ec719f693a3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:49.472489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbe4c5-7584-466b-8078-1ec719f693a3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:49.472489 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbe4c5-7584-466b-8078-1ec719f693a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:49.493324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1dbbe4c5-7584-466b-8078-1ec719f693a3 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:10:49.494655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1dbbe4c5-7584-466b-8078-1ec719f693a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:49.581834 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1155/2304] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:10:49.592523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8ccfb-1646-4983-8bd5-0b832e2b9c84 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:49.592881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8ccfb-1646-4983-8bd5-0b832e2b9c84 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:49.593318 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8ccfb-1646-4983-8bd5-0b832e2b9c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:49.631191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8ab8ccfb-1646-4983-8bd5-0b832e2b9c84 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:10:49.632258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8ab8ccfb-1646-4983-8bd5-0b832e2b9c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:49.644851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1150/2305] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:10:49.649233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbe4c5-7584-466b-8078-1ec719f693a3 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:49.660514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbe4c5-7584-466b-8078-1ec719f693a3 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:49.667413 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b05280-f9b6-4f4a-857b-b86c5b57a586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:49.668117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b05280-f9b6-4f4a-857b-b86c5b57a586 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:49.668464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b05280-f9b6-4f4a-857b-b86c5b57a586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:49.668808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b05280-f9b6-4f4a-857b-b86c5b57a586 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:49.669576 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b05280-f9b6-4f4a-857b-b86c5b57a586 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:10:49.669924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b05280-f9b6-4f4a-857b-b86c5b57a586 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:49.671577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b05280-f9b6-4f4a-857b-b86c5b57a586 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:49.673606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:10:49.688999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1156/2306] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:10:49.693708 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8ccfb-1646-4983-8bd5-0b832e2b9c84 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:49.703801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8ccfb-1646-4983-8bd5-0b832e2b9c84 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:49.708677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5f36f-9269-4695-81cd-233a381a6606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:49.709350 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5f36f-9269-4695-81cd-233a381a6606 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:49.709684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5f36f-9269-4695-81cd-233a381a6606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:49.709964 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5f36f-9269-4695-81cd-233a381a6606 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:49.711432 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5f36f-9269-4695-81cd-233a381a6606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:49.713884 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5f36f-9269-4695-81cd-233a381a6606 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:49.755559 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1151/2307] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:10:49 2022] POST /identity/v3/projects => generated 355 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:10:49.759801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b05280-f9b6-4f4a-857b-b86c5b57a586 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:49.770806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b05280-f9b6-4f4a-857b-b86c5b57a586 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:49.839982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e194dee-00c6-4434-86e2-9594904b3d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:49.840659 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e194dee-00c6-4434-86e2-9594904b3d38 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:49.840909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e194dee-00c6-4434-86e2-9594904b3d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:49.841201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e194dee-00c6-4434-86e2-9594904b3d38 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:49.841930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e194dee-00c6-4434-86e2-9594904b3d38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:49.843888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e194dee-00c6-4434-86e2-9594904b3d38 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:49.867108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1157/2308] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:10:49 2022] POST /identity/v3/users => generated 278 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:10:49.871625 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5f36f-9269-4695-81cd-233a381a6606 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:49.883583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5f36f-9269-4695-81cd-233a381a6606 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:49.888584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ddbe4-e3a1-41d0-9c41-727e432f5afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:49.889391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ddbe4-e3a1-41d0-9c41-727e432f5afa admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:49.889700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ddbe4-e3a1-41d0-9c41-727e432f5afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:49.889974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ddbe4-e3a1-41d0-9c41-727e432f5afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:49.890730 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ddbe4-e3a1-41d0-9c41-727e432f5afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:49.892329 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ddbe4-e3a1-41d0-9c41-727e432f5afa admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:49.903852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1152/2309] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:10:49.908240 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e194dee-00c6-4434-86e2-9594904b3d38 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:49.918332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e194dee-00c6-4434-86e2-9594904b3d38 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:49.923362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfa044-f6ac-4f8f-8b02-1279ea880259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:49.924132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfa044-f6ac-4f8f-8b02-1279ea880259 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:49.924488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfa044-f6ac-4f8f-8b02-1279ea880259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:49.924827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfa044-f6ac-4f8f-8b02-1279ea880259 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:49.925802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfa044-f6ac-4f8f-8b02-1279ea880259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:49.927592 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfa044-f6ac-4f8f-8b02-1279ea880259 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:49.938219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1158/2310] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:10:49.942453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ddbe4-e3a1-41d0-9c41-727e432f5afa admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:49.953818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ddbe4-e3a1-41d0-9c41-727e432f5afa admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:49.960083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452cc07-b67a-4902-a101-52e502dac92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:49.960977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452cc07-b67a-4902-a101-52e502dac92b admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:49.961193 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452cc07-b67a-4902-a101-52e502dac92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:49.961474 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452cc07-b67a-4902-a101-52e502dac92b admin admin] PATH_INFO: `/v3/projects/669cb0f690ef40afa4ad6d55b4227bce/users/59f1d1cccb364042a6e93ddc46db2db6/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:49.998054 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452cc07-b67a-4902-a101-52e502dac92b admin admin] RBAC: Authorizing `identity:create_grant(project_id=669cb0f690ef40afa4ad6d55b4227bce, user_id=59f1d1cccb364042a6e93ddc46db2db6, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:50.000934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452cc07-b67a-4902-a101-52e502dac92b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:50.026237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1153/2311] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:10:49 2022] PUT /identity/v3/projects/669cb0f690ef40afa4ad6d55b4227bce/users/59f1d1cccb364042a6e93ddc46db2db6/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:10:50.037749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440013d3-d4e8-47a6-9563-f57738ec392f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:50.038284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440013d3-d4e8-47a6-9563-f57738ec392f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:50.038566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440013d3-d4e8-47a6-9563-f57738ec392f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:50.083754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-440013d3-d4e8-47a6-9563-f57738ec392f None None] MFA Rules not processed for user `59f1d1cccb364042a6e93ddc46db2db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:10:50.085116 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-440013d3-d4e8-47a6-9563-f57738ec392f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:50.179859 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1159/2312] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:10:50 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:10:50.184574 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452cc07-b67a-4902-a101-52e502dac92b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:50.190505 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:10:50.257523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452cc07-b67a-4902-a101-52e502dac92b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:50.262586 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfbcf6-e7c8-4ec4-866d-6d1edc5f00e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:50.263267 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfbcf6-e7c8-4ec4-866d-6d1edc5f00e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:50.263551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfbcf6-e7c8-4ec4-866d-6d1edc5f00e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:50.263831 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfbcf6-e7c8-4ec4-866d-6d1edc5f00e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:50.264534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfbcf6-e7c8-4ec4-866d-6d1edc5f00e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:50.266515 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfbcf6-e7c8-4ec4-866d-6d1edc5f00e4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:50.294209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1154/2313] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:10:50 2022] POST /identity/v3/projects => generated 357 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:10:50.299326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440013d3-d4e8-47a6-9563-f57738ec392f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:50.311536 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440013d3-d4e8-47a6-9563-f57738ec392f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:50.379579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71527e3-bb71-4c19-8343-42120d985c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:50.380958 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71527e3-bb71-4c19-8343-42120d985c26 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:50.380958 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71527e3-bb71-4c19-8343-42120d985c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:50.381199 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71527e3-bb71-4c19-8343-42120d985c26 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:50.382252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71527e3-bb71-4c19-8343-42120d985c26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:50.383868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71527e3-bb71-4c19-8343-42120d985c26 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:50.392783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:10:50.414652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1160/2314] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:10:50 2022] POST /identity/v3/users => generated 285 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:10:50.418699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfbcf6-e7c8-4ec4-866d-6d1edc5f00e4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:50.432091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfbcf6-e7c8-4ec4-866d-6d1edc5f00e4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:50.440127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04886d50-efab-4889-b85f-b9b423149c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:50.441071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04886d50-efab-4889-b85f-b9b423149c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:50.441380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04886d50-efab-4889-b85f-b9b423149c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:50.442669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04886d50-efab-4889-b85f-b9b423149c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:50.442669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04886d50-efab-4889-b85f-b9b423149c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:50.444060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04886d50-efab-4889-b85f-b9b423149c0a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:50.458999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1155/2315] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:10:50.461049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71527e3-bb71-4c19-8343-42120d985c26 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:50.472573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71527e3-bb71-4c19-8343-42120d985c26 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:50.478376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42df411-6d93-4ba7-9f71-c4aee796faa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:50.479666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42df411-6d93-4ba7-9f71-c4aee796faa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:50.480091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42df411-6d93-4ba7-9f71-c4aee796faa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:50.480387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42df411-6d93-4ba7-9f71-c4aee796faa6 admin admin] PATH_INFO: `/v3/projects/309ec68b8b6f4cd3b2fb3bf3e1c9b8f8/users/4ae0dd7c5b4d4f3091b441fb225d710a/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:50.498890 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42df411-6d93-4ba7-9f71-c4aee796faa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=309ec68b8b6f4cd3b2fb3bf3e1c9b8f8, user_id=4ae0dd7c5b4d4f3091b441fb225d710a, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:50.502888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42df411-6d93-4ba7-9f71-c4aee796faa6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:50.526045 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1161/2316] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:10:50 2022] PUT /identity/v3/projects/309ec68b8b6f4cd3b2fb3bf3e1c9b8f8/users/4ae0dd7c5b4d4f3091b441fb225d710a/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:10:50.530131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04886d50-efab-4889-b85f-b9b423149c0a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:50.542697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04886d50-efab-4889-b85f-b9b423149c0a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:50.614451 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef5ace-32a1-4e49-854c-1d58c3659c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:50.615152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef5ace-32a1-4e49-854c-1d58c3659c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:50.615438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef5ace-32a1-4e49-854c-1d58c3659c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:50.615713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef5ace-32a1-4e49-854c-1d58c3659c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:50.616469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef5ace-32a1-4e49-854c-1d58c3659c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:50.622548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef5ace-32a1-4e49-854c-1d58c3659c2c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:50.636633 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1156/2317] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:10:50.642245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42df411-6d93-4ba7-9f71-c4aee796faa6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:50.655199 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42df411-6d93-4ba7-9f71-c4aee796faa6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:50.660593 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df5ee6-6081-45b6-9414-3caffadb5c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:50.661597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df5ee6-6081-45b6-9414-3caffadb5c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:50.661914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df5ee6-6081-45b6-9414-3caffadb5c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:50.662188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df5ee6-6081-45b6-9414-3caffadb5c8f admin admin] PATH_INFO: `/v3/projects/309ec68b8b6f4cd3b2fb3bf3e1c9b8f8/users/4ae0dd7c5b4d4f3091b441fb225d710a/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:50.664517 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df5ee6-6081-45b6-9414-3caffadb5c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=309ec68b8b6f4cd3b2fb3bf3e1c9b8f8, user_id=4ae0dd7c5b4d4f3091b441fb225d710a, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:50.668048 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df5ee6-6081-45b6-9414-3caffadb5c8f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:50.679967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1162/2318] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:10:50 2022] PUT /identity/v3/projects/309ec68b8b6f4cd3b2fb3bf3e1c9b8f8/users/4ae0dd7c5b4d4f3091b441fb225d710a/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:10:50.691744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44011b-6ccc-48cd-921d-23d44afd4e40 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:50.692126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44011b-6ccc-48cd-921d-23d44afd4e40 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:50.692482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44011b-6ccc-48cd-921d-23d44afd4e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:50.722798 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ec44011b-6ccc-48cd-921d-23d44afd4e40 None None] MFA Rules not processed for user `4ae0dd7c5b4d4f3091b441fb225d710a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:10:50.724461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ec44011b-6ccc-48cd-921d-23d44afd4e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:50.803804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:10:50.809985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1157/2319] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:10:50 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:10:50.820564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df5ee6-6081-45b6-9414-3caffadb5c8f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:50.822473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-48df5ee6-6081-45b6-9414-3caffadb5c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:50.893031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df5ee6-6081-45b6-9414-3caffadb5c8f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:50.897027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c350c-8867-41f5-93ed-b504c1850194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:50.897715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c350c-8867-41f5-93ed-b504c1850194 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:50.897999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c350c-8867-41f5-93ed-b504c1850194 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:50.898285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c350c-8867-41f5-93ed-b504c1850194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:50.909102 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c350c-8867-41f5-93ed-b504c1850194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:50.910923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c350c-8867-41f5-93ed-b504c1850194 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:50.923491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1163/2320] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:10:50 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:10:51.008055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44011b-6ccc-48cd-921d-23d44afd4e40 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:51.021510 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44011b-6ccc-48cd-921d-23d44afd4e40 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:51.035002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67de4f-df79-45bb-8ab4-ee592dc6b83f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:51.039940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67de4f-df79-45bb-8ab4-ee592dc6b83f service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:51.040220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67de4f-df79-45bb-8ab4-ee592dc6b83f service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:51.040501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67de4f-df79-45bb-8ab4-ee592dc6b83f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:51.113732 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67de4f-df79-45bb-8ab4-ee592dc6b83f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:51.114811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67de4f-df79-45bb-8ab4-ee592dc6b83f service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:51.129144 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1158/2321] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:10:51 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:10:53.755942 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c350c-8867-41f5-93ed-b504c1850194 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:53.757356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-761c350c-8867-41f5-93ed-b504c1850194 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:53.828265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c350c-8867-41f5-93ed-b504c1850194 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:53.832089 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42a194-201c-4cba-b6cc-5f2abecb6af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:53.832751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42a194-201c-4cba-b6cc-5f2abecb6af9 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:53.833035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42a194-201c-4cba-b6cc-5f2abecb6af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:53.833496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42a194-201c-4cba-b6cc-5f2abecb6af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:53.835750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4d42a194-201c-4cba-b6cc-5f2abecb6af9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:53.903840 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67de4f-df79-45bb-8ab4-ee592dc6b83f service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:53.913351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42a194-201c-4cba-b6cc-5f2abecb6af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:53.913950 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67de4f-df79-45bb-8ab4-ee592dc6b83f service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:53.917443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42a194-201c-4cba-b6cc-5f2abecb6af9 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:53.919311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8968be6-bcb0-41ab-8315-af6f04d0e70c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:53.920196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8968be6-bcb0-41ab-8315-af6f04d0e70c service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:53.920635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8968be6-bcb0-41ab-8315-af6f04d0e70c service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:53.921082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8968be6-bcb0-41ab-8315-af6f04d0e70c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:53.923468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f8968be6-bcb0-41ab-8315-af6f04d0e70c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:53.934328 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1164/2322] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:10:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:10:53.957584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8968be6-bcb0-41ab-8315-af6f04d0e70c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:53.960192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8968be6-bcb0-41ab-8315-af6f04d0e70c service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:53.974176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1159/2323] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:10:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:10:59.450655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42a194-201c-4cba-b6cc-5f2abecb6af9 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:59.452917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4d42a194-201c-4cba-b6cc-5f2abecb6af9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:59.482290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42a194-201c-4cba-b6cc-5f2abecb6af9 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:59.487989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:59.488907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:59.489225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:59.489452 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] PATH_INFO: `/v3/users/126ba607541e4c4895b47b4b7602ce01` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:59.490933 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] RBAC: Authorizing `identity:delete_user(user_id=126ba607541e4c4895b47b4b7602ce01)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:59.496204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:59.498318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:10:59.578732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126ba607541e4c4895b47b4b7602ce01'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:59.591647 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126ba607541e4c4895b47b4b7602ce01'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:59.601907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126ba607541e4c4895b47b4b7602ce01'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:59.603903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126ba607541e4c4895b47b4b7602ce01'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:59.631773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126ba607541e4c4895b47b4b7602ce01'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:10:59.642698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1165/2324] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:10:59 2022] DELETE /identity/v3/users/126ba607541e4c4895b47b4b7602ce01 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:10:59.795754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8968be6-bcb0-41ab-8315-af6f04d0e70c service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:59.797456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f8968be6-bcb0-41ab-8315-af6f04d0e70c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:59.816111 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8968be6-bcb0-41ab-8315-af6f04d0e70c service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:10:59.853283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:10:59.855333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:10:59.942676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f38e3-0c71-4737-96be-ac60554a4a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:10:59.943980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f38e3-0c71-4737-96be-ac60554a4a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:10:59.944475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f38e3-0c71-4737-96be-ac60554a4a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:10:59.944955 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f38e3-0c71-4737-96be-ac60554a4a83 admin admin] PATH_INFO: `/v3/users/18b1072539af49729dff7ee9ea445c77` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:10:59.946433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f38e3-0c71-4737-96be-ac60554a4a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18b1072539af49729dff7ee9ea445c77)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:10:59.950560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f38e3-0c71-4737-96be-ac60554a4a83 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:10:59.999028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80cdda-c1b8-485b-a2fd-c758e9ca328e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:00.006441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:00.007438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.008075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.008075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0 admin admin] PATH_INFO: `/v3/projects/de5226d001fb47b399172b6633fec844` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.009390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de5226d001fb47b399172b6633fec844)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:00.011522 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:00.042327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-564f38e3-0c71-4737-96be-ac60554a4a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18b1072539af49729dff7ee9ea445c77'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.043097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-564f38e3-0c71-4737-96be-ac60554a4a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18b1072539af49729dff7ee9ea445c77'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.043924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-564f38e3-0c71-4737-96be-ac60554a4a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18b1072539af49729dff7ee9ea445c77'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.061490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-564f38e3-0c71-4737-96be-ac60554a4a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18b1072539af49729dff7ee9ea445c77'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.078175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-564f38e3-0c71-4737-96be-ac60554a4a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18b1072539af49729dff7ee9ea445c77'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.082285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de5226d001fb47b399172b6633fec844', 'request_id': 'req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.088186 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de5226d001fb47b399172b6633fec844', 'request_id': 'req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.097721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1160/2325] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:10:59 2022] DELETE /identity/v3/users/18b1072539af49729dff7ee9ea445c77 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:00.097917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de5226d001fb47b399172b6633fec844', 'request_id': 'req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.114840 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0 admin admin] The token cache is being invalidate because project de5226d001fb47b399172b6633fec844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:00.115138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.115868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef9d6dcf-ef66-47ce-b295-9053f63b4cb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.119550 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1166/2326] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:10:59 2022] DELETE /identity/v3/projects/de5226d001fb47b399172b6633fec844 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:00.124429 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7aba4-3776-46be-bb4a-664b3d309816 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.124753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7aba4-3776-46be-bb4a-664b3d309816 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.125090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7aba4-3776-46be-bb4a-664b3d309816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.134526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:00.150646 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ecd7aba4-3776-46be-bb4a-664b3d309816 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:00.153131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ecd7aba4-3776-46be-bb4a-664b3d309816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:00.230773 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1161/2327] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:00.239128 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5532ed-3d4d-47c0-8a33-f2e55da85ea0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.239432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5532ed-3d4d-47c0-8a33-f2e55da85ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.239683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5532ed-3d4d-47c0-8a33-f2e55da85ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.259604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6e5532ed-3d4d-47c0-8a33-f2e55da85ea0 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:00.260847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6e5532ed-3d4d-47c0-8a33-f2e55da85ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:00.271995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1167/2328] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:00.276523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7aba4-3776-46be-bb4a-664b3d309816 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:00.287702 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7aba4-3776-46be-bb4a-664b3d309816 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:00.296808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5532ed-3d4d-47c0-8a33-f2e55da85ea0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:00.298097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec71fc-8ee7-4032-9aef-226316a26754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:00.298292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6e5532ed-3d4d-47c0-8a33-f2e55da85ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:00.298814 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec71fc-8ee7-4032-9aef-226316a26754 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.299125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec71fc-8ee7-4032-9aef-226316a26754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.299400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec71fc-8ee7-4032-9aef-226316a26754 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.300141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec71fc-8ee7-4032-9aef-226316a26754 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:00.300449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec71fc-8ee7-4032-9aef-226316a26754 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:00.302122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec71fc-8ee7-4032-9aef-226316a26754 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:00.306081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:00.315580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5532ed-3d4d-47c0-8a33-f2e55da85ea0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:00.315875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1162/2329] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:11:00.321219 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec71fc-8ee7-4032-9aef-226316a26754 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:00.323770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95b852-1285-406e-a906-6a1a6e307f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:00.324985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95b852-1285-406e-a906-6a1a6e307f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.325474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95b852-1285-406e-a906-6a1a6e307f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.326234 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95b852-1285-406e-a906-6a1a6e307f68 admin admin] PATH_INFO: `/v3/projects/34a4b394ba684ed2a67d14b4a5b24f2f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.327823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95b852-1285-406e-a906-6a1a6e307f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34a4b394ba684ed2a67d14b4a5b24f2f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:00.330105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95b852-1285-406e-a906-6a1a6e307f68 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:00.337599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec71fc-8ee7-4032-9aef-226316a26754 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:00.343029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47ff81-9649-4652-85e9-edf90cbe1da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:00.343695 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47ff81-9649-4652-85e9-edf90cbe1da0 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.343978 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47ff81-9649-4652-85e9-edf90cbe1da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.344270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47ff81-9649-4652-85e9-edf90cbe1da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.344958 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47ff81-9649-4652-85e9-edf90cbe1da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:00.346910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47ff81-9649-4652-85e9-edf90cbe1da0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:00.387581 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1163/2330] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:00 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:00.392569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47ff81-9649-4652-85e9-edf90cbe1da0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:00.402858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8d95b852-1285-406e-a906-6a1a6e307f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34a4b394ba684ed2a67d14b4a5b24f2f', 'request_id': 'req-8d95b852-1285-406e-a906-6a1a6e307f68'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.412735 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8d95b852-1285-406e-a906-6a1a6e307f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34a4b394ba684ed2a67d14b4a5b24f2f', 'request_id': 'req-8d95b852-1285-406e-a906-6a1a6e307f68'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.426826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8d95b852-1285-406e-a906-6a1a6e307f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34a4b394ba684ed2a67d14b4a5b24f2f', 'request_id': 'req-8d95b852-1285-406e-a906-6a1a6e307f68'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.456727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8d95b852-1285-406e-a906-6a1a6e307f68 admin admin] The token cache is being invalidate because project 34a4b394ba684ed2a67d14b4a5b24f2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:00.457683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8d95b852-1285-406e-a906-6a1a6e307f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.461303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8d95b852-1285-406e-a906-6a1a6e307f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:00.465491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1168/2331] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:00 2022] DELETE /identity/v3/projects/34a4b394ba684ed2a67d14b4a5b24f2f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:00.472112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472ef0d-e513-41ee-acf3-3723b5154bba None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.472616 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472ef0d-e513-41ee-acf3-3723b5154bba None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.472996 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472ef0d-e513-41ee-acf3-3723b5154bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.493330 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6472ef0d-e513-41ee-acf3-3723b5154bba None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:00.494983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6472ef0d-e513-41ee-acf3-3723b5154bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:00.504276 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1169/2332] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:00.507996 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47ff81-9649-4652-85e9-edf90cbe1da0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:00.513239 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090621f-8875-4d86-aeb4-96725d08c9fe None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.513528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090621f-8875-4d86-aeb4-96725d08c9fe None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.513850 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090621f-8875-4d86-aeb4-96725d08c9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.516310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed1bb2-49b4-4730-97d4-d4fa458d3d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:00.517468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed1bb2-49b4-4730-97d4-d4fa458d3d57 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.517998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed1bb2-49b4-4730-97d4-d4fa458d3d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.518518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed1bb2-49b4-4730-97d4-d4fa458d3d57 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.519814 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed1bb2-49b4-4730-97d4-d4fa458d3d57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:00.523748 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed1bb2-49b4-4730-97d4-d4fa458d3d57 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:00.533854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e090621f-8875-4d86-aeb4-96725d08c9fe None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:00.535997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e090621f-8875-4d86-aeb4-96725d08c9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:00.554304 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1170/2333] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:00.555775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090621f-8875-4d86-aeb4-96725d08c9fe None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:00.557861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1164/2334] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:00 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:00.562194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed1bb2-49b4-4730-97d4-d4fa458d3d57 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:00.564336 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e4ed1bb2-49b4-4730-97d4-d4fa458d3d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:00.568693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090621f-8875-4d86-aeb4-96725d08c9fe None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:00.574642 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9b0ad-7f4b-4262-a9c4-3480ea91efae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:00.575914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9b0ad-7f4b-4262-a9c4-3480ea91efae admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.576416 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9b0ad-7f4b-4262-a9c4-3480ea91efae admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.576915 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9b0ad-7f4b-4262-a9c4-3480ea91efae admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.578464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9b0ad-7f4b-4262-a9c4-3480ea91efae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:00.579259 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9b0ad-7f4b-4262-a9c4-3480ea91efae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:00.580754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed1bb2-49b4-4730-97d4-d4fa458d3d57 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:00.582590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9b0ad-7f4b-4262-a9c4-3480ea91efae admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:00.590430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26d58-96b4-49b5-bb5b-3b58922551c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:00.591570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26d58-96b4-49b5-bb5b-3b58922551c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.591896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26d58-96b4-49b5-bb5b-3b58922551c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.592532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26d58-96b4-49b5-bb5b-3b58922551c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.593578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee26d58-96b4-49b5-bb5b-3b58922551c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:00.595408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee26d58-96b4-49b5-bb5b-3b58922551c3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:00.601216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1171/2335] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:11:00.602647 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9b0ad-7f4b-4262-a9c4-3480ea91efae admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:00.609690 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1165/2336] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:11:00.613367 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26d58-96b4-49b5-bb5b-3b58922551c3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:00.620211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9b0ad-7f4b-4262-a9c4-3480ea91efae admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:00.626263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63394353-dd3d-49d5-8bc7-c26bbb07cb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:00.627117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26d58-96b4-49b5-bb5b-3b58922551c3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:00.627251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63394353-dd3d-49d5-8bc7-c26bbb07cb9a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.627961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63394353-dd3d-49d5-8bc7-c26bbb07cb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.628367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63394353-dd3d-49d5-8bc7-c26bbb07cb9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.629181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63394353-dd3d-49d5-8bc7-c26bbb07cb9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:00.631609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63394353-dd3d-49d5-8bc7-c26bbb07cb9a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:00.633617 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831a264-d6da-497c-80d5-6c29676c01d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:00.634921 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831a264-d6da-497c-80d5-6c29676c01d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.635369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831a264-d6da-497c-80d5-6c29676c01d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.635819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831a264-d6da-497c-80d5-6c29676c01d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.637293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831a264-d6da-497c-80d5-6c29676c01d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:00.639876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831a264-d6da-497c-80d5-6c29676c01d4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:00.658797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1166/2337] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:11:00.660045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831a264-d6da-497c-80d5-6c29676c01d4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:00.666365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1172/2338] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:00 2022] POST /identity/v3/projects => generated 403 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:00.670246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63394353-dd3d-49d5-8bc7-c26bbb07cb9a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:00.681592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831a264-d6da-497c-80d5-6c29676c01d4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:00.730158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:00.769149 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3b2e4-4900-47a4-8cd0-66b4a04ace8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:00.770421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3b2e4-4900-47a4-8cd0-66b4a04ace8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.770720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63394353-dd3d-49d5-8bc7-c26bbb07cb9a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:00.771037 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3b2e4-4900-47a4-8cd0-66b4a04ace8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.771469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3b2e4-4900-47a4-8cd0-66b4a04ace8e admin admin] PATH_INFO: `/v3/projects/457cb5c290ec41f7bdddc8e97b76288f/users/8ac3a9d78c844e0199ce6ee58e1ff9e6/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.777853 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a96d0-b923-4b21-b592-7765d7cd4a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:00.778681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a96d0-b923-4b21-b592-7765d7cd4a68 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.779063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a96d0-b923-4b21-b592-7765d7cd4a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.779412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a96d0-b923-4b21-b592-7765d7cd4a68 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.780494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a96d0-b923-4b21-b592-7765d7cd4a68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:00.783470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a96d0-b923-4b21-b592-7765d7cd4a68 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:00.810076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3b2e4-4900-47a4-8cd0-66b4a04ace8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=457cb5c290ec41f7bdddc8e97b76288f, user_id=8ac3a9d78c844e0199ce6ee58e1ff9e6, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:00.814698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3b2e4-4900-47a4-8cd0-66b4a04ace8e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:00.820846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1173/2339] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:00 2022] POST /identity/v3/users => generated 302 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:00.825769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a96d0-b923-4b21-b592-7765d7cd4a68 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:00.839904 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1167/2340] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:00 2022] PUT /identity/v3/projects/457cb5c290ec41f7bdddc8e97b76288f/users/8ac3a9d78c844e0199ce6ee58e1ff9e6/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:00.851019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d8dfa-f64a-4c51-a872-70773004b134 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:00.851313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d8dfa-f64a-4c51-a872-70773004b134 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:00.852129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d8dfa-f64a-4c51-a872-70773004b134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:00.882014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a96d0-b923-4b21-b592-7765d7cd4a68 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:00.900837 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a57d8dfa-f64a-4c51-a872-70773004b134 None None] MFA Rules not processed for user `8ac3a9d78c844e0199ce6ee58e1ff9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:00.906700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a57d8dfa-f64a-4c51-a872-70773004b134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:00.965210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:01.044059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f272219-b632-42e6-a579-b322a87a1ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:01.045065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f272219-b632-42e6-a579-b322a87a1ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:01.045332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f272219-b632-42e6-a579-b322a87a1ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:01.045609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1168/2341] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:01.045965 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f272219-b632-42e6-a579-b322a87a1ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:01.046521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f272219-b632-42e6-a579-b322a87a1ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:01.052633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f272219-b632-42e6-a579-b322a87a1ab2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:01.076413 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d8dfa-f64a-4c51-a872-70773004b134 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:01.085720 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a57d8dfa-f64a-4c51-a872-70773004b134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:01.091523 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1174/2342] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:01.094273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f272219-b632-42e6-a579-b322a87a1ab2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:01.123770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f272219-b632-42e6-a579-b322a87a1ab2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:01.138324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bdd47-05f1-49bb-87dd-b6fb9456da68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:01.139032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bdd47-05f1-49bb-87dd-b6fb9456da68 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:01.139323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bdd47-05f1-49bb-87dd-b6fb9456da68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:01.139600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bdd47-05f1-49bb-87dd-b6fb9456da68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:01.140522 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bdd47-05f1-49bb-87dd-b6fb9456da68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:01.146959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bdd47-05f1-49bb-87dd-b6fb9456da68 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:01.168323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1175/2343] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:01.170310 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bdd47-05f1-49bb-87dd-b6fb9456da68 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:01.183461 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bdd47-05f1-49bb-87dd-b6fb9456da68 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:01.190833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089ce9e-0e4b-4ccd-8293-9ff9d6ba5346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:01.192031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089ce9e-0e4b-4ccd-8293-9ff9d6ba5346 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:01.192314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089ce9e-0e4b-4ccd-8293-9ff9d6ba5346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:01.192650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089ce9e-0e4b-4ccd-8293-9ff9d6ba5346 admin admin] PATH_INFO: `/v3/projects/29a9445aae0a4a3eba40862e33621f44/users/9beef16950134f19bbc32ea875220c7a/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:01.217131 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089ce9e-0e4b-4ccd-8293-9ff9d6ba5346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a9445aae0a4a3eba40862e33621f44, user_id=9beef16950134f19bbc32ea875220c7a, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:01.220161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089ce9e-0e4b-4ccd-8293-9ff9d6ba5346 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:01.258291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1176/2344] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:01 2022] PUT /identity/v3/projects/29a9445aae0a4a3eba40862e33621f44/users/9beef16950134f19bbc32ea875220c7a/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:01.259532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d8dfa-f64a-4c51-a872-70773004b134 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:01.263749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77737d1b-e186-43ae-8f38-8c9e0fff9e9d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:01.264776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77737d1b-e186-43ae-8f38-8c9e0fff9e9d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:01.266326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77737d1b-e186-43ae-8f38-8c9e0fff9e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:01.325270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-77737d1b-e186-43ae-8f38-8c9e0fff9e9d None None] MFA Rules not processed for user `9beef16950134f19bbc32ea875220c7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:01.327526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:01.329483 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-77737d1b-e186-43ae-8f38-8c9e0fff9e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:01.366889 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ece77c-c952-41b8-af1f-8dac18f5ea97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:01.368294 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ece77c-c952-41b8-af1f-8dac18f5ea97 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:01.368709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ece77c-c952-41b8-af1f-8dac18f5ea97 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:01.369200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ece77c-c952-41b8-af1f-8dac18f5ea97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:01.457772 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1177/2345] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:01.462238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77737d1b-e186-43ae-8f38-8c9e0fff9e9d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:01.476084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ece77c-c952-41b8-af1f-8dac18f5ea97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:01.479668 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ece77c-c952-41b8-af1f-8dac18f5ea97 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:01.490809 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1169/2346] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:01.618318 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77737d1b-e186-43ae-8f38-8c9e0fff9e9d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:01.638886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111be389-c49d-4edc-9740-ae87fdfbb60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:01.641430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111be389-c49d-4edc-9740-ae87fdfbb60d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:01.641725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111be389-c49d-4edc-9740-ae87fdfbb60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:01.643038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111be389-c49d-4edc-9740-ae87fdfbb60d admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:01.644274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111be389-c49d-4edc-9740-ae87fdfbb60d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:01.647957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111be389-c49d-4edc-9740-ae87fdfbb60d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:01.718758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ece77c-c952-41b8-af1f-8dac18f5ea97 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:01.733775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1178/2347] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:01 2022] POST /identity/v3/projects => generated 403 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:01.739980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ece77c-c952-41b8-af1f-8dac18f5ea97 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:01.860657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4009a-e352-4fe7-909f-47e5c96aea94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:01.862409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4009a-e352-4fe7-909f-47e5c96aea94 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:01.862864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4009a-e352-4fe7-909f-47e5c96aea94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:01.863130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4009a-e352-4fe7-909f-47e5c96aea94 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:01.863882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4009a-e352-4fe7-909f-47e5c96aea94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:01.866102 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4009a-e352-4fe7-909f-47e5c96aea94 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:01.905332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1170/2348] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:01 2022] POST /identity/v3/users => generated 308 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:01.909256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111be389-c49d-4edc-9740-ae87fdfbb60d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:01.916249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:01.925937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111be389-c49d-4edc-9740-ae87fdfbb60d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:01.933155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0476477-16b0-4ff6-8d72-2af503601d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:01.934068 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0476477-16b0-4ff6-8d72-2af503601d65 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:01.934474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0476477-16b0-4ff6-8d72-2af503601d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:01.934787 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0476477-16b0-4ff6-8d72-2af503601d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:01.935684 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0476477-16b0-4ff6-8d72-2af503601d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:01.939119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0476477-16b0-4ff6-8d72-2af503601d65 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:01.952405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1179/2349] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:01.956764 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4009a-e352-4fe7-909f-47e5c96aea94 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:01.968882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4009a-e352-4fe7-909f-47e5c96aea94 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:01.978372 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cce59-5bc0-49a3-a209-0b7ad99f32bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:01.980382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cce59-5bc0-49a3-a209-0b7ad99f32bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:01.982517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cce59-5bc0-49a3-a209-0b7ad99f32bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:01.983197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cce59-5bc0-49a3-a209-0b7ad99f32bb admin admin] PATH_INFO: `/v3/projects/3ecb1a8e57074d5b86d995ea3f73a887/users/03a16137bd6a40c3ba5deb09254a0e7d/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:02.008472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cce59-5bc0-49a3-a209-0b7ad99f32bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ecb1a8e57074d5b86d995ea3f73a887, user_id=03a16137bd6a40c3ba5deb09254a0e7d, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:02.015158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cce59-5bc0-49a3-a209-0b7ad99f32bb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:02.057843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1171/2350] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:01 2022] PUT /identity/v3/projects/3ecb1a8e57074d5b86d995ea3f73a887/users/03a16137bd6a40c3ba5deb09254a0e7d/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:02.062878 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0476477-16b0-4ff6-8d72-2af503601d65 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:02.079313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0476477-16b0-4ff6-8d72-2af503601d65 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:02.163763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1207b3-c97b-4310-b68e-5f4de32db2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:02.164724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1207b3-c97b-4310-b68e-5f4de32db2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:02.164724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1207b3-c97b-4310-b68e-5f4de32db2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:02.165132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1207b3-c97b-4310-b68e-5f4de32db2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:02.165996 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1207b3-c97b-4310-b68e-5f4de32db2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:02.167651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1207b3-c97b-4310-b68e-5f4de32db2d9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:02.179051 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1180/2351] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:11:02.184062 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cce59-5bc0-49a3-a209-0b7ad99f32bb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:02.195530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cce59-5bc0-49a3-a209-0b7ad99f32bb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:02.200466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758016c-73d7-49d7-b9d3-f48b51ac7943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:02.201376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758016c-73d7-49d7-b9d3-f48b51ac7943 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:02.201676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758016c-73d7-49d7-b9d3-f48b51ac7943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:02.201952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758016c-73d7-49d7-b9d3-f48b51ac7943 admin admin] PATH_INFO: `/v3/projects/3ecb1a8e57074d5b86d995ea3f73a887/users/03a16137bd6a40c3ba5deb09254a0e7d/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:02.204114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758016c-73d7-49d7-b9d3-f48b51ac7943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ecb1a8e57074d5b86d995ea3f73a887, user_id=03a16137bd6a40c3ba5deb09254a0e7d, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:02.206813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758016c-73d7-49d7-b9d3-f48b51ac7943 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:02.218484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1172/2352] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:02 2022] PUT /identity/v3/projects/3ecb1a8e57074d5b86d995ea3f73a887/users/03a16137bd6a40c3ba5deb09254a0e7d/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:02.229821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6f2db-2601-49fa-a10e-0a98dbd65d54 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:02.230223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6f2db-2601-49fa-a10e-0a98dbd65d54 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:02.231258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6f2db-2601-49fa-a10e-0a98dbd65d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:02.262099 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-46f6f2db-2601-49fa-a10e-0a98dbd65d54 None None] MFA Rules not processed for user `03a16137bd6a40c3ba5deb09254a0e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:02.263394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-46f6f2db-2601-49fa-a10e-0a98dbd65d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:02.345560 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1181/2353] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:02.358378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758016c-73d7-49d7-b9d3-f48b51ac7943 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:02.360402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f758016c-73d7-49d7-b9d3-f48b51ac7943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:02.408402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:02.433529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758016c-73d7-49d7-b9d3-f48b51ac7943 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:02.437246 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c59628-1f74-4632-93eb-f5e87334d46b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:02.437878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c59628-1f74-4632-93eb-f5e87334d46b service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:02.438163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c59628-1f74-4632-93eb-f5e87334d46b service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:02.438442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c59628-1f74-4632-93eb-f5e87334d46b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:02.448460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c59628-1f74-4632-93eb-f5e87334d46b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:02.449943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c59628-1f74-4632-93eb-f5e87334d46b service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:02.461282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1173/2354] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:02 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:02.616129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6f2db-2601-49fa-a10e-0a98dbd65d54 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:02.693396 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6f2db-2601-49fa-a10e-0a98dbd65d54 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:02.699934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:02.701014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:02.701383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:02.701726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] PATH_INFO: `/v3/users/9beef16950134f19bbc32ea875220c7a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:02.703210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9beef16950134f19bbc32ea875220c7a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:02.705563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:02.723545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:02.765364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9beef16950134f19bbc32ea875220c7a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:02.775438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9beef16950134f19bbc32ea875220c7a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:02.783445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9beef16950134f19bbc32ea875220c7a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:02.784139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9beef16950134f19bbc32ea875220c7a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:02.797963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9beef16950134f19bbc32ea875220c7a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:02.804668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1182/2355] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:02 2022] DELETE /identity/v3/users/9beef16950134f19bbc32ea875220c7a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:02.810106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a83e68-af93-44c7-b60a-a0b2f4b27606 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:02.810465 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a83e68-af93-44c7-b60a-a0b2f4b27606 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:02.810660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a83e68-af93-44c7-b60a-a0b2f4b27606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:02.830615 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d9a83e68-af93-44c7-b60a-a0b2f4b27606 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:02.831958 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d9a83e68-af93-44c7-b60a-a0b2f4b27606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:02.913362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1174/2356] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:02.923923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:02.926408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:03.001390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d1437-4419-49a8-95a3-bb19e343f08b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:03.005694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327ebda-1566-43a0-bc78-bb71ed487e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:03.006423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327ebda-1566-43a0-bc78-bb71ed487e8f service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:03.006711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327ebda-1566-43a0-bc78-bb71ed487e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:03.006985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327ebda-1566-43a0-bc78-bb71ed487e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:03.017163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327ebda-1566-43a0-bc78-bb71ed487e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:03.018676 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327ebda-1566-43a0-bc78-bb71ed487e8f service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:03.031649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1183/2357] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:03.360183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a83e68-af93-44c7-b60a-a0b2f4b27606 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:03.361807 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d9a83e68-af93-44c7-b60a-a0b2f4b27606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:03.375398 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a83e68-af93-44c7-b60a-a0b2f4b27606 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:03.385413 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:03.386374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:03.388013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:03.388359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] PATH_INFO: `/v3/projects/29a9445aae0a4a3eba40862e33621f44` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:03.392447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a9445aae0a4a3eba40862e33621f44)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:03.399485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:03.451381 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a9445aae0a4a3eba40862e33621f44', 'request_id': 'req-b04cc901-adef-4cfa-95cf-9fdf0074238a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:03.457957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a9445aae0a4a3eba40862e33621f44', 'request_id': 'req-b04cc901-adef-4cfa-95cf-9fdf0074238a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:03.464687 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a9445aae0a4a3eba40862e33621f44', 'request_id': 'req-b04cc901-adef-4cfa-95cf-9fdf0074238a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:03.478782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] The token cache is being invalidate because project 29a9445aae0a4a3eba40862e33621f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:03.479079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:03.479694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:03.481303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1175/2358] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:03 2022] DELETE /identity/v3/projects/29a9445aae0a4a3eba40862e33621f44 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:03.485654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327ebda-1566-43a0-bc78-bb71ed487e8f service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:03.496106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327ebda-1566-43a0-bc78-bb71ed487e8f service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:03.559547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87862d09-236c-43b7-b70e-7446db5d83dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:03.560517 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87862d09-236c-43b7-b70e-7446db5d83dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:03.560799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87862d09-236c-43b7-b70e-7446db5d83dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:03.561075 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87862d09-236c-43b7-b70e-7446db5d83dc admin admin] PATH_INFO: `/v3/users/03a16137bd6a40c3ba5deb09254a0e7d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:03.562524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87862d09-236c-43b7-b70e-7446db5d83dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a16137bd6a40c3ba5deb09254a0e7d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:03.564619 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87862d09-236c-43b7-b70e-7446db5d83dc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:03.611575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:03.624139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-87862d09-236c-43b7-b70e-7446db5d83dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a16137bd6a40c3ba5deb09254a0e7d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:03.633279 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-87862d09-236c-43b7-b70e-7446db5d83dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a16137bd6a40c3ba5deb09254a0e7d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:03.641808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-87862d09-236c-43b7-b70e-7446db5d83dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a16137bd6a40c3ba5deb09254a0e7d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:03.642603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-87862d09-236c-43b7-b70e-7446db5d83dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a16137bd6a40c3ba5deb09254a0e7d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:03.657606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-87862d09-236c-43b7-b70e-7446db5d83dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a16137bd6a40c3ba5deb09254a0e7d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:03.664566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1184/2359] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:03 2022] DELETE /identity/v3/users/03a16137bd6a40c3ba5deb09254a0e7d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:04.001718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:04.004918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:04.028094 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cc901-adef-4cfa-95cf-9fdf0074238a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:04.097323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:04.098265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.098517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.098789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6 admin admin] PATH_INFO: `/v3/projects/3ecb1a8e57074d5b86d995ea3f73a887` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.099990 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ecb1a8e57074d5b86d995ea3f73a887)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:04.102017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:04.109954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:04.161053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ecb1a8e57074d5b86d995ea3f73a887', 'request_id': 'req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:04.168662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ecb1a8e57074d5b86d995ea3f73a887', 'request_id': 'req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:04.176077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ecb1a8e57074d5b86d995ea3f73a887', 'request_id': 'req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:04.191832 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6 admin admin] The token cache is being invalidate because project 3ecb1a8e57074d5b86d995ea3f73a887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:04.192281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:04.192972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a45ca595-a9bc-4ae0-bf74-e0b37a0b51d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:04.194617 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1176/2360] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:04 2022] DELETE /identity/v3/projects/3ecb1a8e57074d5b86d995ea3f73a887 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:04.199958 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1878092-eb7b-4b92-9f6f-5c1a13ca9e9d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.200246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1878092-eb7b-4b92-9f6f-5c1a13ca9e9d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.200523 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1878092-eb7b-4b92-9f6f-5c1a13ca9e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.225743 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c1878092-eb7b-4b92-9f6f-5c1a13ca9e9d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:04.227303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c1878092-eb7b-4b92-9f6f-5c1a13ca9e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:04.313379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1185/2361] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:04.321746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35cd6c-07ff-4265-b492-8e41fb4304bd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.322238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35cd6c-07ff-4265-b492-8e41fb4304bd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.322718 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35cd6c-07ff-4265-b492-8e41fb4304bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.347895 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2d35cd6c-07ff-4265-b492-8e41fb4304bd None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:04.349114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2d35cd6c-07ff-4265-b492-8e41fb4304bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:04.357960 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1177/2362] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:04.362670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1878092-eb7b-4b92-9f6f-5c1a13ca9e9d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:04.376197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1878092-eb7b-4b92-9f6f-5c1a13ca9e9d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:04.382082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6927cc-0820-41f9-b622-425e990ce1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:04.382777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6927cc-0820-41f9-b622-425e990ce1a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.383059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6927cc-0820-41f9-b622-425e990ce1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.383335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6927cc-0820-41f9-b622-425e990ce1a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.384456 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6927cc-0820-41f9-b622-425e990ce1a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:04.384772 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6927cc-0820-41f9-b622-425e990ce1a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:04.387202 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6927cc-0820-41f9-b622-425e990ce1a4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:04.401493 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1186/2363] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:11:04.406230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35cd6c-07ff-4265-b492-8e41fb4304bd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:04.418307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35cd6c-07ff-4265-b492-8e41fb4304bd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:04.423575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0083d7-86ec-4c85-a4bb-545190533662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:04.424168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0083d7-86ec-4c85-a4bb-545190533662 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.424453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0083d7-86ec-4c85-a4bb-545190533662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.424797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0083d7-86ec-4c85-a4bb-545190533662 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.425481 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0083d7-86ec-4c85-a4bb-545190533662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:04.427927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0083d7-86ec-4c85-a4bb-545190533662 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:04.467906 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1178/2364] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:04 2022] POST /identity/v3/projects => generated 403 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:04.470948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6927cc-0820-41f9-b622-425e990ce1a4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:04.476863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0083d7-86ec-4c85-a4bb-545190533662 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:04.481657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6927cc-0820-41f9-b622-425e990ce1a4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:04.482657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2b0083d7-86ec-4c85-a4bb-545190533662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:04.569427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaf279-6a0a-459c-b130-2694c4e96ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:04.570277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaf279-6a0a-459c-b130-2694c4e96ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.570707 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaf279-6a0a-459c-b130-2694c4e96ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.571195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaf279-6a0a-459c-b130-2694c4e96ed6 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.572580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaf279-6a0a-459c-b130-2694c4e96ed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:04.574628 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaf279-6a0a-459c-b130-2694c4e96ed6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:04.574819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0083d7-86ec-4c85-a4bb-545190533662 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:04.579647 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbae2c3-881c-4c90-a8f5-90bd92a3db3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:04.580541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbae2c3-881c-4c90-a8f5-90bd92a3db3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.580820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbae2c3-881c-4c90-a8f5-90bd92a3db3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.581098 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbae2c3-881c-4c90-a8f5-90bd92a3db3e admin admin] PATH_INFO: `/v3/users/8ac3a9d78c844e0199ce6ee58e1ff9e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.582582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbae2c3-881c-4c90-a8f5-90bd92a3db3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ac3a9d78c844e0199ce6ee58e1ff9e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:04.584626 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbae2c3-881c-4c90-a8f5-90bd92a3db3e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:04.600399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1187/2365] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:04 2022] POST /identity/v3/users => generated 302 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:04.605017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaf279-6a0a-459c-b130-2694c4e96ed6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:04.617299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaf279-6a0a-459c-b130-2694c4e96ed6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:04.623805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f9748-cedf-4681-898d-5cbc6137b489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:04.624809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f9748-cedf-4681-898d-5cbc6137b489 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.625577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f9748-cedf-4681-898d-5cbc6137b489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.625577 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f9748-cedf-4681-898d-5cbc6137b489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.626209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f9748-cedf-4681-898d-5cbc6137b489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:04.628083 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f9748-cedf-4681-898d-5cbc6137b489 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:04.643156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1188/2366] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:11:04.644373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f9748-cedf-4681-898d-5cbc6137b489 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:04.646164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dfbae2c3-881c-4c90-a8f5-90bd92a3db3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ac3a9d78c844e0199ce6ee58e1ff9e6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:04.646949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dfbae2c3-881c-4c90-a8f5-90bd92a3db3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ac3a9d78c844e0199ce6ee58e1ff9e6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:04.647809 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dfbae2c3-881c-4c90-a8f5-90bd92a3db3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ac3a9d78c844e0199ce6ee58e1ff9e6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:04.655556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f9748-cedf-4681-898d-5cbc6137b489 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:04.659423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dfbae2c3-881c-4c90-a8f5-90bd92a3db3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ac3a9d78c844e0199ce6ee58e1ff9e6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:04.660695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0164444-aeab-4119-9a6d-bbe084526b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:04.661818 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0164444-aeab-4119-9a6d-bbe084526b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.662114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0164444-aeab-4119-9a6d-bbe084526b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.662394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0164444-aeab-4119-9a6d-bbe084526b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.663137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0164444-aeab-4119-9a6d-bbe084526b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:04.664863 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0164444-aeab-4119-9a6d-bbe084526b2a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:04.669754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:04.675887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dfbae2c3-881c-4c90-a8f5-90bd92a3db3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ac3a9d78c844e0199ce6ee58e1ff9e6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:04.676625 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1189/2367] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:04.681428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0164444-aeab-4119-9a6d-bbe084526b2a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:04.683274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f0164444-aeab-4119-9a6d-bbe084526b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:04.691413 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1179/2368] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:04 2022] DELETE /identity/v3/users/8ac3a9d78c844e0199ce6ee58e1ff9e6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:04.696344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe6bcc-33fa-416a-a772-1a5ccedda5b8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.696607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe6bcc-33fa-416a-a772-1a5ccedda5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.696887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe6bcc-33fa-416a-a772-1a5ccedda5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.699094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0164444-aeab-4119-9a6d-bbe084526b2a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:04.704306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:04.716176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a5fe6bcc-33fa-416a-a772-1a5ccedda5b8 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:04.717566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a5fe6bcc-33fa-416a-a772-1a5ccedda5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:04.773855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4907a-95a2-4336-9253-d61230811c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:04.775123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4907a-95a2-4336-9253-d61230811c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.775487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4907a-95a2-4336-9253-d61230811c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.775740 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4907a-95a2-4336-9253-d61230811c3e admin admin] PATH_INFO: `/v3/projects/5e0f3ec313bc4b5ca263c5565f0a4dac/users/8e14eb1276a54442bb5d1ab5cf82eb03/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.793250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1180/2369] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:04.797390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4907a-95a2-4336-9253-d61230811c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e0f3ec313bc4b5ca263c5565f0a4dac, user_id=8e14eb1276a54442bb5d1ab5cf82eb03, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:04.800265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4907a-95a2-4336-9253-d61230811c3e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:04.803146 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe6bcc-33fa-416a-a772-1a5ccedda5b8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:04.804511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a5fe6bcc-33fa-416a-a772-1a5ccedda5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:04.818934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1190/2370] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:04 2022] PUT /identity/v3/projects/5e0f3ec313bc4b5ca263c5565f0a4dac/users/8e14eb1276a54442bb5d1ab5cf82eb03/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:04.825077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a87c60-caa5-4ac5-bb2f-d0dfa2dfe735 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.825367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a87c60-caa5-4ac5-bb2f-d0dfa2dfe735 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.825676 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a87c60-caa5-4ac5-bb2f-d0dfa2dfe735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.844760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-65a87c60-caa5-4ac5-bb2f-d0dfa2dfe735 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:04.846059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-65a87c60-caa5-4ac5-bb2f-d0dfa2dfe735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:04.878313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe6bcc-33fa-416a-a772-1a5ccedda5b8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:04.919036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1191/2371] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:04.929156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a87c60-caa5-4ac5-bb2f-d0dfa2dfe735 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:04.931548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-65a87c60-caa5-4ac5-bb2f-d0dfa2dfe735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:04.934849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58132910-7d5b-4abf-936e-ebfbbb401b67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:04.935520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58132910-7d5b-4abf-936e-ebfbbb401b67 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.935800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58132910-7d5b-4abf-936e-ebfbbb401b67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.936071 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58132910-7d5b-4abf-936e-ebfbbb401b67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.944353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a87c60-caa5-4ac5-bb2f-d0dfa2dfe735 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:04.946107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58132910-7d5b-4abf-936e-ebfbbb401b67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:04.947734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58132910-7d5b-4abf-936e-ebfbbb401b67 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:04.948885 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410e2e7-ab69-4df6-86e3-d142f3cec4ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:04.949554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410e2e7-ab69-4df6-86e3-d142f3cec4ae service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:04.949866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410e2e7-ab69-4df6-86e3-d142f3cec4ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:04.950142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410e2e7-ab69-4df6-86e3-d142f3cec4ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:04.960076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1181/2372] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:04.964339 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410e2e7-ab69-4df6-86e3-d142f3cec4ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:04.966166 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410e2e7-ab69-4df6-86e3-d142f3cec4ae service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:04.979060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1192/2373] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:05.083251 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58132910-7d5b-4abf-936e-ebfbbb401b67 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:05.095153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58132910-7d5b-4abf-936e-ebfbbb401b67 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:05.100376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b054e2f-4d14-42b4-b30f-c1b83c500f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:05.101190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b054e2f-4d14-42b4-b30f-c1b83c500f58 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:05.101477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b054e2f-4d14-42b4-b30f-c1b83c500f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:05.101826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b054e2f-4d14-42b4-b30f-c1b83c500f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:05.110256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3b054e2f-4d14-42b4-b30f-c1b83c500f58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:05.126655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b054e2f-4d14-42b4-b30f-c1b83c500f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:05.129420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b054e2f-4d14-42b4-b30f-c1b83c500f58 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:05.143774 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1182/2374] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:05.317795 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410e2e7-ab69-4df6-86e3-d142f3cec4ae service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:05.319309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8410e2e7-ab69-4df6-86e3-d142f3cec4ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:05.345774 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410e2e7-ab69-4df6-86e3-d142f3cec4ae service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:05.351709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:05.354213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:05.354606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:05.354960 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25 admin admin] PATH_INFO: `/v3/projects/457cb5c290ec41f7bdddc8e97b76288f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:05.356300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=457cb5c290ec41f7bdddc8e97b76288f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:05.358226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:05.433132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '457cb5c290ec41f7bdddc8e97b76288f', 'request_id': 'req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:05.444488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '457cb5c290ec41f7bdddc8e97b76288f', 'request_id': 'req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:05.460235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '457cb5c290ec41f7bdddc8e97b76288f', 'request_id': 'req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:05.478532 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25 admin admin] The token cache is being invalidate because project 457cb5c290ec41f7bdddc8e97b76288f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:05.479112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:05.480077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-30b4ad7b-6e25-4457-8992-cbf46d1dfc25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:05.483032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1193/2375] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:05 2022] DELETE /identity/v3/projects/457cb5c290ec41f7bdddc8e97b76288f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:05.489051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c60321-c6b4-4e6f-924e-ff19573f2950 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:05.489564 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c60321-c6b4-4e6f-924e-ff19573f2950 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:05.490063 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c60321-c6b4-4e6f-924e-ff19573f2950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:05.512925 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-65c60321-c6b4-4e6f-924e-ff19573f2950 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:05.515348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-65c60321-c6b4-4e6f-924e-ff19573f2950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:05.561140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:05.612834 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1183/2376] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:05.620694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819eaca4-8fd1-418a-baf2-6da0c1206665 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:05.621241 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819eaca4-8fd1-418a-baf2-6da0c1206665 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:05.621712 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819eaca4-8fd1-418a-baf2-6da0c1206665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:05.629970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:05.668017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-819eaca4-8fd1-418a-baf2-6da0c1206665 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:05.672097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-819eaca4-8fd1-418a-baf2-6da0c1206665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:05.684373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1194/2377] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:05.689035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c60321-c6b4-4e6f-924e-ff19573f2950 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:05.704475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c60321-c6b4-4e6f-924e-ff19573f2950 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:05.723644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341ef86-6124-4296-8e50-3deb895fabf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:05.724348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341ef86-6124-4296-8e50-3deb895fabf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:05.724687 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341ef86-6124-4296-8e50-3deb895fabf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:05.724907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341ef86-6124-4296-8e50-3deb895fabf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:05.725871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341ef86-6124-4296-8e50-3deb895fabf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:05.726357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341ef86-6124-4296-8e50-3deb895fabf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:05.728989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341ef86-6124-4296-8e50-3deb895fabf6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:05.763382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1184/2378] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:11:05.768209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819eaca4-8fd1-418a-baf2-6da0c1206665 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:05.782146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819eaca4-8fd1-418a-baf2-6da0c1206665 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:05.787451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9675f20-5c1e-40f2-9ed4-823253d5d85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:05.788617 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9675f20-5c1e-40f2-9ed4-823253d5d85b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:05.789058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9675f20-5c1e-40f2-9ed4-823253d5d85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:05.789491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9675f20-5c1e-40f2-9ed4-823253d5d85b admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:05.791154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9675f20-5c1e-40f2-9ed4-823253d5d85b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:05.793504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9675f20-5c1e-40f2-9ed4-823253d5d85b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:05.846981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1195/2379] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:05 2022] POST /identity/v3/projects => generated 367 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:05.847742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341ef86-6124-4296-8e50-3deb895fabf6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:05.860664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341ef86-6124-4296-8e50-3deb895fabf6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:05.974128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c00284-25c7-474e-b863-4fb730cdd128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:05.974926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c00284-25c7-474e-b863-4fb730cdd128 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:05.975125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c00284-25c7-474e-b863-4fb730cdd128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:05.975412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c00284-25c7-474e-b863-4fb730cdd128 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:05.977841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c00284-25c7-474e-b863-4fb730cdd128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:05.981552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c00284-25c7-474e-b863-4fb730cdd128 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:06.026707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1185/2380] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:05 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:06.028499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9675f20-5c1e-40f2-9ed4-823253d5d85b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:06.046296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9675f20-5c1e-40f2-9ed4-823253d5d85b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:06.050767 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c63cbd-5732-448d-8f8e-eac7a982548c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:06.051103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c63cbd-5732-448d-8f8e-eac7a982548c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:06.051439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c63cbd-5732-448d-8f8e-eac7a982548c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:06.054133 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b421ee4-9f92-4f6c-88d1-f7ec7ded56f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:06.064233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b421ee4-9f92-4f6c-88d1-f7ec7ded56f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:06.064513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b421ee4-9f92-4f6c-88d1-f7ec7ded56f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:06.065030 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b421ee4-9f92-4f6c-88d1-f7ec7ded56f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:06.066054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b421ee4-9f92-4f6c-88d1-f7ec7ded56f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:06.070494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b421ee4-9f92-4f6c-88d1-f7ec7ded56f5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:06.090242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1196/2381] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:06.091415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b421ee4-9f92-4f6c-88d1-f7ec7ded56f5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:06.117226 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b421ee4-9f92-4f6c-88d1-f7ec7ded56f5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:06.119456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b4c63cbd-5732-448d-8f8e-eac7a982548c None None] MFA Rules not processed for user `8e14eb1276a54442bb5d1ab5cf82eb03`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:06.121254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b4c63cbd-5732-448d-8f8e-eac7a982548c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:06.127228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c474494-e4fb-4e8d-ac6e-6176487f5fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:06.128267 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c474494-e4fb-4e8d-ac6e-6176487f5fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:06.128550 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c474494-e4fb-4e8d-ac6e-6176487f5fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:06.129582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c474494-e4fb-4e8d-ac6e-6176487f5fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:06.129909 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c474494-e4fb-4e8d-ac6e-6176487f5fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:06.132233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c474494-e4fb-4e8d-ac6e-6176487f5fc3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:06.151435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1197/2382] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:06.155711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c474494-e4fb-4e8d-ac6e-6176487f5fc3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:06.170113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c474494-e4fb-4e8d-ac6e-6176487f5fc3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:06.178763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d254d-cabd-4c53-985b-6d998decefdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:06.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d254d-cabd-4c53-985b-6d998decefdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:06.180962 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d254d-cabd-4c53-985b-6d998decefdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:06.181788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d254d-cabd-4c53-985b-6d998decefdf admin admin] PATH_INFO: `/v3/projects/f7a1fb14d292479387cb49af03570e3a/users/7c1e3814919144b6af526b1c0b3890a8/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:06.212109 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d254d-cabd-4c53-985b-6d998decefdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a1fb14d292479387cb49af03570e3a, user_id=7c1e3814919144b6af526b1c0b3890a8, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:06.213788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1186/2383] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:06.214916 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d254d-cabd-4c53-985b-6d998decefdf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:06.215276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c63cbd-5732-448d-8f8e-eac7a982548c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:06.216689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b4c63cbd-5732-448d-8f8e-eac7a982548c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:06.234693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c63cbd-5732-448d-8f8e-eac7a982548c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:06.247077 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1198/2384] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:06 2022] PUT /identity/v3/projects/f7a1fb14d292479387cb49af03570e3a/users/7c1e3814919144b6af526b1c0b3890a8/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:06.259527 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbedd6-cbb2-48e0-ab4f-182849d9c124 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:06.259929 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbedd6-cbb2-48e0-ab4f-182849d9c124 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:06.260289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbedd6-cbb2-48e0-ab4f-182849d9c124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:06.288809 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:06.300548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fccbedd6-cbb2-48e0-ab4f-182849d9c124 None None] MFA Rules not processed for user `7c1e3814919144b6af526b1c0b3890a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:06.302063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fccbedd6-cbb2-48e0-ab4f-182849d9c124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:06.338077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d666fd8-72fe-42d1-99bf-cb50fe78b183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:06.339490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d666fd8-72fe-42d1-99bf-cb50fe78b183 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:06.340128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d666fd8-72fe-42d1-99bf-cb50fe78b183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:06.340558 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d666fd8-72fe-42d1-99bf-cb50fe78b183 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:06.342028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d666fd8-72fe-42d1-99bf-cb50fe78b183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:06.345003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d666fd8-72fe-42d1-99bf-cb50fe78b183 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:06.380475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1187/2385] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:06 2022] POST /identity/v3/projects => generated 405 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:06.385202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d666fd8-72fe-42d1-99bf-cb50fe78b183 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:06.396711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1199/2386] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:06.408293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbedd6-cbb2-48e0-ab4f-182849d9c124 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:06.411044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fccbedd6-cbb2-48e0-ab4f-182849d9c124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:06.508411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d666fd8-72fe-42d1-99bf-cb50fe78b183 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:06.517141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af305f7-1e1f-4dd9-ba59-f1252aa44600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:06.518371 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af305f7-1e1f-4dd9-ba59-f1252aa44600 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:06.518881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af305f7-1e1f-4dd9-ba59-f1252aa44600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:06.519487 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af305f7-1e1f-4dd9-ba59-f1252aa44600 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:06.520455 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af305f7-1e1f-4dd9-ba59-f1252aa44600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:06.522800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af305f7-1e1f-4dd9-ba59-f1252aa44600 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:06.522968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbedd6-cbb2-48e0-ab4f-182849d9c124 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:06.554819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1188/2387] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:06 2022] POST /identity/v3/users => generated 309 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:06.563107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af305f7-1e1f-4dd9-ba59-f1252aa44600 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:06.576805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af305f7-1e1f-4dd9-ba59-f1252aa44600 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:06.580162 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:06.594120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae04be-20cf-4dab-92b4-95a757cc186c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:06.594878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae04be-20cf-4dab-92b4-95a757cc186c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:06.595167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae04be-20cf-4dab-92b4-95a757cc186c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:06.595510 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae04be-20cf-4dab-92b4-95a757cc186c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:06.596402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae04be-20cf-4dab-92b4-95a757cc186c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:06.598613 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae04be-20cf-4dab-92b4-95a757cc186c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:06.613848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1189/2388] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:06.615207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf0b0f-b06d-41d2-9623-90586dd16e33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:06.615939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0b0f-b06d-41d2-9623-90586dd16e33 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:06.616273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0b0f-b06d-41d2-9623-90586dd16e33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:06.617005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0b0f-b06d-41d2-9623-90586dd16e33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:06.618173 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae04be-20cf-4dab-92b4-95a757cc186c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:06.630354 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae04be-20cf-4dab-92b4-95a757cc186c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:06.635956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd33fc-cc29-4d0d-8aeb-81c4406d450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:06.637210 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd33fc-cc29-4d0d-8aeb-81c4406d450e admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:06.637604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd33fc-cc29-4d0d-8aeb-81c4406d450e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:06.637950 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd33fc-cc29-4d0d-8aeb-81c4406d450e admin admin] PATH_INFO: `/v3/projects/2fcfc2823d314f389fda173abce9b452/users/bccb5e9d78d94d1188f1d161c2aa0f36/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:06.666366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd33fc-cc29-4d0d-8aeb-81c4406d450e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fcfc2823d314f389fda173abce9b452, user_id=bccb5e9d78d94d1188f1d161c2aa0f36, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:06.669980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd33fc-cc29-4d0d-8aeb-81c4406d450e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:06.692044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1190/2389] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:06 2022] PUT /identity/v3/projects/2fcfc2823d314f389fda173abce9b452/users/bccb5e9d78d94d1188f1d161c2aa0f36/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:06.694152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd33fc-cc29-4d0d-8aeb-81c4406d450e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:06.714726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf0b0f-b06d-41d2-9623-90586dd16e33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:06.718502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf0b0f-b06d-41d2-9623-90586dd16e33 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:06.746223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1200/2390] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:06.785473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd33fc-cc29-4d0d-8aeb-81c4406d450e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:06.792299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f20036-0ac7-4af8-82c6-b0755cb6c7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:06.792672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f20036-0ac7-4af8-82c6-b0755cb6c7cb admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:06.793063 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f20036-0ac7-4af8-82c6-b0755cb6c7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:06.793325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f20036-0ac7-4af8-82c6-b0755cb6c7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:06.794237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f20036-0ac7-4af8-82c6-b0755cb6c7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:06.796089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f20036-0ac7-4af8-82c6-b0755cb6c7cb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:06.808695 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1191/2391] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:11:06.813670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf0b0f-b06d-41d2-9623-90586dd16e33 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:06.826198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf0b0f-b06d-41d2-9623-90586dd16e33 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:06.831224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a18ba-7845-4e9b-9bf1-ef51d7bba46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:06.832156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a18ba-7845-4e9b-9bf1-ef51d7bba46e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:06.832511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a18ba-7845-4e9b-9bf1-ef51d7bba46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:06.832758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a18ba-7845-4e9b-9bf1-ef51d7bba46e admin admin] PATH_INFO: `/v3/projects/2fcfc2823d314f389fda173abce9b452/users/bccb5e9d78d94d1188f1d161c2aa0f36/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:06.835101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a18ba-7845-4e9b-9bf1-ef51d7bba46e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fcfc2823d314f389fda173abce9b452, user_id=bccb5e9d78d94d1188f1d161c2aa0f36, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:06.838672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a18ba-7845-4e9b-9bf1-ef51d7bba46e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:06.854704 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1201/2392] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:06 2022] PUT /identity/v3/projects/2fcfc2823d314f389fda173abce9b452/users/bccb5e9d78d94d1188f1d161c2aa0f36/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:07.521152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f20036-0ac7-4af8-82c6-b0755cb6c7cb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:07.523607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-41f20036-0ac7-4af8-82c6-b0755cb6c7cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:07.539923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f20036-0ac7-4af8-82c6-b0755cb6c7cb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:07.555498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:07.640925 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadcbfd-7d54-43f1-a449-b347a9dd2959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:07.642117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadcbfd-7d54-43f1-a449-b347a9dd2959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:07.642567 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadcbfd-7d54-43f1-a449-b347a9dd2959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:07.642876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadcbfd-7d54-43f1-a449-b347a9dd2959 admin admin] PATH_INFO: `/v3/users/59f1d1cccb364042a6e93ddc46db2db6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:07.645397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffadcbfd-7d54-43f1-a449-b347a9dd2959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59f1d1cccb364042a6e93ddc46db2db6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:07.649176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffadcbfd-7d54-43f1-a449-b347a9dd2959 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:07.739677 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ffadcbfd-7d54-43f1-a449-b347a9dd2959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59f1d1cccb364042a6e93ddc46db2db6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:07.740550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ffadcbfd-7d54-43f1-a449-b347a9dd2959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59f1d1cccb364042a6e93ddc46db2db6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:07.742287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ffadcbfd-7d54-43f1-a449-b347a9dd2959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59f1d1cccb364042a6e93ddc46db2db6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:07.753324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ffadcbfd-7d54-43f1-a449-b347a9dd2959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59f1d1cccb364042a6e93ddc46db2db6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:07.770534 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ffadcbfd-7d54-43f1-a449-b347a9dd2959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59f1d1cccb364042a6e93ddc46db2db6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:07.794755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1192/2393] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:07 2022] DELETE /identity/v3/users/59f1d1cccb364042a6e93ddc46db2db6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:07.802502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fea40-bff1-418c-a0f1-2bd304e9eda2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:07.803250 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fea40-bff1-418c-a0f1-2bd304e9eda2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:07.803250 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fea40-bff1-418c-a0f1-2bd304e9eda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:07.844415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-972fea40-bff1-418c-a0f1-2bd304e9eda2 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:07.848302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-972fea40-bff1-418c-a0f1-2bd304e9eda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:07.902155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca278a59-ad38-4b91-9e28-598ccfb073ec None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:07.902500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca278a59-ad38-4b91-9e28-598ccfb073ec None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:07.902900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca278a59-ad38-4b91-9e28-598ccfb073ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:07.943631 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ca278a59-ad38-4b91-9e28-598ccfb073ec None None] MFA Rules not processed for user `bccb5e9d78d94d1188f1d161c2aa0f36`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:07.944971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ca278a59-ad38-4b91-9e28-598ccfb073ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:08.006380 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1202/2394] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:08.016717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fea40-bff1-418c-a0f1-2bd304e9eda2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:08.018716 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-972fea40-bff1-418c-a0f1-2bd304e9eda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:08.056947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1193/2395] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:07 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:08.122846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fea40-bff1-418c-a0f1-2bd304e9eda2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:08.135546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c693f-8e52-43af-acf3-701a26168400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:08.137969 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c693f-8e52-43af-acf3-701a26168400 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:08.138812 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c693f-8e52-43af-acf3-701a26168400 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:08.139730 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c693f-8e52-43af-acf3-701a26168400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:08.168983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c693f-8e52-43af-acf3-701a26168400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:08.170615 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c693f-8e52-43af-acf3-701a26168400 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:08.183232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1203/2396] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:09.175065 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca278a59-ad38-4b91-9e28-598ccfb073ec None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:09.177174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ca278a59-ad38-4b91-9e28-598ccfb073ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:09.202860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca278a59-ad38-4b91-9e28-598ccfb073ec None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:09.215192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:09.216802 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:09.217346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:09.218368 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] PATH_INFO: `/v3/projects/669cb0f690ef40afa4ad6d55b4227bce` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:09.220042 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=669cb0f690ef40afa4ad6d55b4227bce)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:09.222910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:09.261688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:09.356119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '669cb0f690ef40afa4ad6d55b4227bce', 'request_id': 'req-6f3b1d04-6041-4d99-9751-ce4ce57e3098'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:09.362620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '669cb0f690ef40afa4ad6d55b4227bce', 'request_id': 'req-6f3b1d04-6041-4d99-9751-ce4ce57e3098'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:09.374706 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '669cb0f690ef40afa4ad6d55b4227bce', 'request_id': 'req-6f3b1d04-6041-4d99-9751-ce4ce57e3098'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:09.407117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] The token cache is being invalidate because project 669cb0f690ef40afa4ad6d55b4227bce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:09.407624 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:09.408746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:09.412289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1194/2397] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:09 2022] DELETE /identity/v3/projects/669cb0f690ef40afa4ad6d55b4227bce => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:09.417086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c693f-8e52-43af-acf3-701a26168400 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:09.430611 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c693f-8e52-43af-acf3-701a26168400 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:09.454593 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:09.520391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:09.521559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:09.522032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:09.522462 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] PATH_INFO: `/v3/users/4ae0dd7c5b4d4f3091b441fb225d710a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:09.523936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ae0dd7c5b4d4f3091b441fb225d710a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:09.526382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:09.644023 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ae0dd7c5b4d4f3091b441fb225d710a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:09.663351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ae0dd7c5b4d4f3091b441fb225d710a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:09.685523 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ae0dd7c5b4d4f3091b441fb225d710a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:09.689222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ae0dd7c5b4d4f3091b441fb225d710a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:09.718524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ae0dd7c5b4d4f3091b441fb225d710a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:09.728229 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1204/2398] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:09 2022] DELETE /identity/v3/users/4ae0dd7c5b4d4f3091b441fb225d710a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:10.129544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:10.132501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:10.156100 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b1d04-6041-4d99-9751-ce4ce57e3098 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:10.331077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:10.333671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:10.334187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:10.334619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] PATH_INFO: `/v3/users/1501eb69e0a447ce83baa5bdee770d1f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:10.337699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1501eb69e0a447ce83baa5bdee770d1f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:10.341280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:10.377353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:10.379867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:10.405115 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec0834-5755-44c4-9cd6-91fb56ce2f8e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:10.414526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:10.417972 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:10.418947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:10.418947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200 admin admin] PATH_INFO: `/v3/projects/309ec68b8b6f4cd3b2fb3bf3e1c9b8f8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:10.420360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=309ec68b8b6f4cd3b2fb3bf3e1c9b8f8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:10.423273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:10.483998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1501eb69e0a447ce83baa5bdee770d1f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.486270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1501eb69e0a447ce83baa5bdee770d1f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.487575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1501eb69e0a447ce83baa5bdee770d1f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.510013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1501eb69e0a447ce83baa5bdee770d1f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.545928 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '309ec68b8b6f4cd3b2fb3bf3e1c9b8f8', 'request_id': 'req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.556245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1501eb69e0a447ce83baa5bdee770d1f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.560984 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '309ec68b8b6f4cd3b2fb3bf3e1c9b8f8', 'request_id': 'req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.579184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '309ec68b8b6f4cd3b2fb3bf3e1c9b8f8', 'request_id': 'req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.581685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1195/2399] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:10 2022] DELETE /identity/v3/users/1501eb69e0a447ce83baa5bdee770d1f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:10.582683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:10.585085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:10.607120 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200 admin admin] The token cache is being invalidate because project 309ec68b8b6f4cd3b2fb3bf3e1c9b8f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:10.607794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.608864 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-98bcc091-e8b7-4d37-b64f-d5c9aaf35200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.612627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1205/2400] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:10 2022] DELETE /identity/v3/projects/309ec68b8b6f4cd3b2fb3bf3e1c9b8f8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:10.622339 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11151335-5111-4a77-a48d-6a4dcd988fc1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:10.623154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11151335-5111-4a77-a48d-6a4dcd988fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:10.625181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11151335-5111-4a77-a48d-6a4dcd988fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:10.631462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:10.664158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-11151335-5111-4a77-a48d-6a4dcd988fc1 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:10.666398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-11151335-5111-4a77-a48d-6a4dcd988fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:10.690970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:10.691424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d043f5-9c3a-4902-8c86-aed39ef69465 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:10.697160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:10.698460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:10.698943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:10.699415 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] PATH_INFO: `/v3/users/8e14eb1276a54442bb5d1ab5cf82eb03` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:10.701486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e14eb1276a54442bb5d1ab5cf82eb03)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:10.707625 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:10.783376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1206/2401] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:10.786891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e14eb1276a54442bb5d1ab5cf82eb03'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.788190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e14eb1276a54442bb5d1ab5cf82eb03'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.788901 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e14eb1276a54442bb5d1ab5cf82eb03'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.794236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f882e-0505-4464-816a-b3be73a1e9c4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:10.794690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f882e-0505-4464-816a-b3be73a1e9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:10.795216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f882e-0505-4464-816a-b3be73a1e9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:10.802731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e14eb1276a54442bb5d1ab5cf82eb03'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.822800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ac0f882e-0505-4464-816a-b3be73a1e9c4 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:10.824656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ac0f882e-0505-4464-816a-b3be73a1e9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:10.826396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e14eb1276a54442bb5d1ab5cf82eb03'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:10.835896 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1207/2402] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:10.841177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f882e-0505-4464-816a-b3be73a1e9c4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:10.855692 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1196/2403] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:10 2022] DELETE /identity/v3/users/8e14eb1276a54442bb5d1ab5cf82eb03 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:10.857864 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f882e-0505-4464-816a-b3be73a1e9c4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:10.916774 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:10.918610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:11.001532 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec59ca4-7707-4336-bec6-1b9fbe105738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:11.003124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec59ca4-7707-4336-bec6-1b9fbe105738 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.003697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec59ca4-7707-4336-bec6-1b9fbe105738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.003982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec59ca4-7707-4336-bec6-1b9fbe105738 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.005021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec59ca4-7707-4336-bec6-1b9fbe105738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:11.006295 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec59ca4-7707-4336-bec6-1b9fbe105738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:11.009033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec59ca4-7707-4336-bec6-1b9fbe105738 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:11.030135 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1208/2404] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:11:11.035194 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec59ca4-7707-4336-bec6-1b9fbe105738 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:11.043403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76f5d0-20d5-4def-9927-11ffaa145a88 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:11.049802 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:11.050811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.051163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.051340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec59ca4-7707-4336-bec6-1b9fbe105738 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:11.051579 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] PATH_INFO: `/v3/projects/4fadf5b15f7e43b4bd2372f22b293518` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.053906 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fadf5b15f7e43b4bd2372f22b293518)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:11.058520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:11.075088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568103ea-b595-4374-8c28-d1d6e6c3388c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:11.075975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568103ea-b595-4374-8c28-d1d6e6c3388c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.076293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568103ea-b595-4374-8c28-d1d6e6c3388c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.076728 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568103ea-b595-4374-8c28-d1d6e6c3388c admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.077453 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568103ea-b595-4374-8c28-d1d6e6c3388c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:11.079520 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568103ea-b595-4374-8c28-d1d6e6c3388c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:11.132070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:11.137904 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1209/2405] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:11 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:11.137904 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568103ea-b595-4374-8c28-d1d6e6c3388c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:11.138681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-568103ea-b595-4374-8c28-d1d6e6c3388c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:11.145287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fadf5b15f7e43b4bd2372f22b293518', 'request_id': 'req-217bbf83-6609-4f5b-a041-b9d5e09256e6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.163330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fadf5b15f7e43b4bd2372f22b293518', 'request_id': 'req-217bbf83-6609-4f5b-a041-b9d5e09256e6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.172155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fadf5b15f7e43b4bd2372f22b293518', 'request_id': 'req-217bbf83-6609-4f5b-a041-b9d5e09256e6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.188555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] The token cache is being invalidate because project 4fadf5b15f7e43b4bd2372f22b293518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:11.188766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.189401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.191186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1197/2406] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:10 2022] DELETE /identity/v3/projects/4fadf5b15f7e43b4bd2372f22b293518 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:11.195901 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:11.195901 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:11.232582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568103ea-b595-4374-8c28-d1d6e6c3388c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:11.239138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:11.240427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.240818 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.241219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041 admin admin] PATH_INFO: `/v3/projects/5e0f3ec313bc4b5ca263c5565f0a4dac` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.243003 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e0f3ec313bc4b5ca263c5565f0a4dac)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:11.246097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:11.285081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:11.296033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bbf83-6609-4f5b-a041-b9d5e09256e6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:11.304625 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e0f3ec313bc4b5ca263c5565f0a4dac', 'request_id': 'req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.311009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e0f3ec313bc4b5ca263c5565f0a4dac', 'request_id': 'req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.330874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e0f3ec313bc4b5ca263c5565f0a4dac', 'request_id': 'req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.348687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041 admin admin] The token cache is being invalidate because project 5e0f3ec313bc4b5ca263c5565f0a4dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:11.349270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.350093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ab0ec8d8-8eaf-4270-a6d6-cec244c27041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.351897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1210/2407] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:11 2022] DELETE /identity/v3/projects/5e0f3ec313bc4b5ca263c5565f0a4dac => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:11.354155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264e8ab-e842-490c-b98c-acd265f10062 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.354637 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264e8ab-e842-490c-b98c-acd265f10062 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.355621 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264e8ab-e842-490c-b98c-acd265f10062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.379544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2264e8ab-e842-490c-b98c-acd265f10062 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:11.381428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2264e8ab-e842-490c-b98c-acd265f10062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:11.389952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3efa11-7419-4ba6-9a65-6c97aef3209b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:11.390882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3efa11-7419-4ba6-9a65-6c97aef3209b admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.391006 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3efa11-7419-4ba6-9a65-6c97aef3209b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.391220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3efa11-7419-4ba6-9a65-6c97aef3209b admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.391962 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3efa11-7419-4ba6-9a65-6c97aef3209b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:11.394064 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3efa11-7419-4ba6-9a65-6c97aef3209b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:11.394775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1211/2408] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:11.396189 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264e8ab-e842-490c-b98c-acd265f10062 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:11.398501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2264e8ab-e842-490c-b98c-acd265f10062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:11.417652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264e8ab-e842-490c-b98c-acd265f10062 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:11.425689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:11.427150 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.427631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.428045 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] PATH_INFO: `/v3/users/bccb5e9d78d94d1188f1d161c2aa0f36` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.430403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1198/2409] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:11 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:11.430640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bccb5e9d78d94d1188f1d161c2aa0f36)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:11.432722 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af111e7e-a449-4933-b4aa-4bfe0ead4f91 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.433186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af111e7e-a449-4933-b4aa-4bfe0ead4f91 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.433647 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af111e7e-a449-4933-b4aa-4bfe0ead4f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.433855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:11.460287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-af111e7e-a449-4933-b4aa-4bfe0ead4f91 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:11.462510 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-af111e7e-a449-4933-b4aa-4bfe0ead4f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:11.478276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1199/2410] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:11.479992 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af111e7e-a449-4933-b4aa-4bfe0ead4f91 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:11.482728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-af111e7e-a449-4933-b4aa-4bfe0ead4f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:11.503603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af111e7e-a449-4933-b4aa-4bfe0ead4f91 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:11.504996 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bccb5e9d78d94d1188f1d161c2aa0f36'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.512091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb129eb7-e1d7-4c07-9825-db174dbccc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:11.513143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb129eb7-e1d7-4c07-9825-db174dbccc38 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.513510 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb129eb7-e1d7-4c07-9825-db174dbccc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.514016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb129eb7-e1d7-4c07-9825-db174dbccc38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.515267 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb129eb7-e1d7-4c07-9825-db174dbccc38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:11.517724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bccb5e9d78d94d1188f1d161c2aa0f36'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.518941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb129eb7-e1d7-4c07-9825-db174dbccc38 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:11.526860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bccb5e9d78d94d1188f1d161c2aa0f36'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.527694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bccb5e9d78d94d1188f1d161c2aa0f36'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.539096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1200/2411] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:11.540896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb129eb7-e1d7-4c07-9825-db174dbccc38 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:11.542946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cb129eb7-e1d7-4c07-9825-db174dbccc38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:11.546842 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bccb5e9d78d94d1188f1d161c2aa0f36'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:11.559093 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb129eb7-e1d7-4c07-9825-db174dbccc38 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:11.561690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1212/2412] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:11 2022] DELETE /identity/v3/users/bccb5e9d78d94d1188f1d161c2aa0f36 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:11.562881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:11.564487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:11.565014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a7261-2645-4669-a49f-39c88bc5fc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:11.566309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a7261-2645-4669-a49f-39c88bc5fc06 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.566309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a7261-2645-4669-a49f-39c88bc5fc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.566653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a7261-2645-4669-a49f-39c88bc5fc06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.567430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a7261-2645-4669-a49f-39c88bc5fc06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:11.567810 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a7261-2645-4669-a49f-39c88bc5fc06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:11.569604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a7261-2645-4669-a49f-39c88bc5fc06 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:11.602170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1201/2413] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:11:11.604698 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a7261-2645-4669-a49f-39c88bc5fc06 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:11.618309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a7261-2645-4669-a49f-39c88bc5fc06 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:11.680451 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:11.686931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712caaa-5f8b-458d-8089-91893718e312 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:11.693668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b01f7-58c0-4170-a855-e03f987c93ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:11.694393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b01f7-58c0-4170-a855-e03f987c93ca admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.694681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b01f7-58c0-4170-a855-e03f987c93ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.695363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b01f7-58c0-4170-a855-e03f987c93ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.696183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b01f7-58c0-4170-a855-e03f987c93ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:11.698231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b01f7-58c0-4170-a855-e03f987c93ca admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:11.712093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1213/2414] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:11.716809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b01f7-58c0-4170-a855-e03f987c93ca admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:11.726943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe10038-87c5-4396-bce2-8006da579044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:11.727956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe10038-87c5-4396-bce2-8006da579044 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.728314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe10038-87c5-4396-bce2-8006da579044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.728660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe10038-87c5-4396-bce2-8006da579044 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.729141 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b01f7-58c0-4170-a855-e03f987c93ca admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:11.729559 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe10038-87c5-4396-bce2-8006da579044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:11.734037 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe10038-87c5-4396-bce2-8006da579044 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:11.735127 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b1f9-e250-45ae-8c0e-36e99c52b46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:11.736217 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b1f9-e250-45ae-8c0e-36e99c52b46c admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.736512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b1f9-e250-45ae-8c0e-36e99c52b46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.736803 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b1f9-e250-45ae-8c0e-36e99c52b46c admin admin] PATH_INFO: `/v3/projects/32f42b4279cf4029ba2d5f077117a90a/users/80de7c9f3ca64664b1c9920a6eae5222/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.759079 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b1f9-e250-45ae-8c0e-36e99c52b46c admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f42b4279cf4029ba2d5f077117a90a, user_id=80de7c9f3ca64664b1c9920a6eae5222, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:11.762231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b1f9-e250-45ae-8c0e-36e99c52b46c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:11.765064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:11.797327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1214/2415] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:11 2022] PUT /identity/v3/projects/32f42b4279cf4029ba2d5f077117a90a/users/80de7c9f3ca64664b1c9920a6eae5222/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:11.817084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3a3b2-d95a-40cb-a824-a3d3f2fe9113 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.817681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3a3b2-d95a-40cb-a824-a3d3f2fe9113 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.818283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3a3b2-d95a-40cb-a824-a3d3f2fe9113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.821589 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1202/2416] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:11 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:11.822681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe10038-87c5-4396-bce2-8006da579044 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:11.871556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-74c3a3b2-d95a-40cb-a824-a3d3f2fe9113 None None] MFA Rules not processed for user `80de7c9f3ca64664b1c9920a6eae5222`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:11.873058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-74c3a3b2-d95a-40cb-a824-a3d3f2fe9113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:11.947954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe10038-87c5-4396-bce2-8006da579044 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:11.954878 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884fa28-fe5f-4274-9a10-fa199a8105bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:11.955713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884fa28-fe5f-4274-9a10-fa199a8105bf admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:11.955879 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884fa28-fe5f-4274-9a10-fa199a8105bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:11.956144 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884fa28-fe5f-4274-9a10-fa199a8105bf admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:11.956938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884fa28-fe5f-4274-9a10-fa199a8105bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:11.958862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884fa28-fe5f-4274-9a10-fa199a8105bf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:11.989657 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1203/2418] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:11 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:11.989898 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1215/2418] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:11.990008 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884fa28-fe5f-4274-9a10-fa199a8105bf admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:11.990273 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f884fa28-fe5f-4274-9a10-fa199a8105bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:11.991743 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3a3b2-d95a-40cb-a824-a3d3f2fe9113 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.007275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3a3b2-d95a-40cb-a824-a3d3f2fe9113 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.010264 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884fa28-fe5f-4274-9a10-fa199a8105bf admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.012890 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc660440-1ba2-48db-b506-8904b51f8aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.013858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc660440-1ba2-48db-b506-8904b51f8aae admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.014449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc660440-1ba2-48db-b506-8904b51f8aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.014829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc660440-1ba2-48db-b506-8904b51f8aae admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.015487 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44413498-912e-4772-942b-5592142377d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.015693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc660440-1ba2-48db-b506-8904b51f8aae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.016595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44413498-912e-4772-942b-5592142377d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.016936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44413498-912e-4772-942b-5592142377d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.017324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44413498-912e-4772-942b-5592142377d5 admin admin] PATH_INFO: `/v3/projects/2fcfc2823d314f389fda173abce9b452` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.018214 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc660440-1ba2-48db-b506-8904b51f8aae admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.018712 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44413498-912e-4772-942b-5592142377d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fcfc2823d314f389fda173abce9b452)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.020719 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44413498-912e-4772-942b-5592142377d5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.056785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1216/2419] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:11 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:12.056785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc660440-1ba2-48db-b506-8904b51f8aae admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.083734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-44413498-912e-4772-942b-5592142377d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fcfc2823d314f389fda173abce9b452', 'request_id': 'req-44413498-912e-4772-942b-5592142377d5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:12.091650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-44413498-912e-4772-942b-5592142377d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fcfc2823d314f389fda173abce9b452', 'request_id': 'req-44413498-912e-4772-942b-5592142377d5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:12.099872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-44413498-912e-4772-942b-5592142377d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fcfc2823d314f389fda173abce9b452', 'request_id': 'req-44413498-912e-4772-942b-5592142377d5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:12.122588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-44413498-912e-4772-942b-5592142377d5 admin admin] The token cache is being invalidate because project 2fcfc2823d314f389fda173abce9b452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:12.122894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-44413498-912e-4772-942b-5592142377d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:12.124437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-44413498-912e-4772-942b-5592142377d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:12.126445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1204/2420] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:11 2022] DELETE /identity/v3/projects/2fcfc2823d314f389fda173abce9b452 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:12.131304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44413498-912e-4772-942b-5592142377d5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.131304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-44413498-912e-4772-942b-5592142377d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:12.148720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc660440-1ba2-48db-b506-8904b51f8aae admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.186182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:12.220203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44413498-912e-4772-942b-5592142377d5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.224116 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f1c34-4595-4fb7-8f0d-42f73850abd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.224834 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f1c34-4595-4fb7-8f0d-42f73850abd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.225120 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f1c34-4595-4fb7-8f0d-42f73850abd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.225374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbdd4a2-340a-401f-9933-87f3f11d41aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.225510 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f1c34-4595-4fb7-8f0d-42f73850abd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.226085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbdd4a2-340a-401f-9933-87f3f11d41aa admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.226373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbdd4a2-340a-401f-9933-87f3f11d41aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.226495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f1c34-4595-4fb7-8f0d-42f73850abd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.226659 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbdd4a2-340a-401f-9933-87f3f11d41aa admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.227446 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbdd4a2-340a-401f-9933-87f3f11d41aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.228556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f1c34-4595-4fb7-8f0d-42f73850abd9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.229578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbdd4a2-340a-401f-9933-87f3f11d41aa admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.239967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1217/2421] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:12.245582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ef8f7-441e-4685-9669-3278fb8869c9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.245582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ef8f7-441e-4685-9669-3278fb8869c9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.245810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ef8f7-441e-4685-9669-3278fb8869c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.263359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1205/2422] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:12 2022] POST /identity/v3/users => generated 297 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:12.263359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbdd4a2-340a-401f-9933-87f3f11d41aa admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.263359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8fbdd4a2-340a-401f-9933-87f3f11d41aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:12.271271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbdd4a2-340a-401f-9933-87f3f11d41aa admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.272588 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ee8ef8f7-441e-4685-9669-3278fb8869c9 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:12.273939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ee8ef8f7-441e-4685-9669-3278fb8869c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:12.274687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:12.276256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301e4ca-c008-49af-a510-57a25a0ea287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.276943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301e4ca-c008-49af-a510-57a25a0ea287 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.277247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301e4ca-c008-49af-a510-57a25a0ea287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.277516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301e4ca-c008-49af-a510-57a25a0ea287 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.278348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301e4ca-c008-49af-a510-57a25a0ea287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.280020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301e4ca-c008-49af-a510-57a25a0ea287 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.292103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1206/2423] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:12.292994 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301e4ca-c008-49af-a510-57a25a0ea287 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.304385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1218/2424] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:12.307256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301e4ca-c008-49af-a510-57a25a0ea287 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.308119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ef8f7-441e-4685-9669-3278fb8869c9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.314279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcb533-2870-4cd6-a0d5-5a43a7287607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.315091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcb533-2870-4cd6-a0d5-5a43a7287607 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.315470 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcb533-2870-4cd6-a0d5-5a43a7287607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.315817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcb533-2870-4cd6-a0d5-5a43a7287607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.316628 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcb533-2870-4cd6-a0d5-5a43a7287607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.318419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcb533-2870-4cd6-a0d5-5a43a7287607 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.325025 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ef8f7-441e-4685-9669-3278fb8869c9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.330987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b4ebf-026f-4591-a38d-dab4f66ecf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.336027 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b4ebf-026f-4591-a38d-dab4f66ecf19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.336203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1207/2425] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:12.336325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b4ebf-026f-4591-a38d-dab4f66ecf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.336325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b4ebf-026f-4591-a38d-dab4f66ecf19 admin admin] PATH_INFO: `/v3/projects/f07c8b9747cf48ada26a23f2d13768f6/users/d2601c7687184bb49d3476d3faff85a1/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.336498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540a4ff-179d-4f4e-ac78-84b527bea2c4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.336498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540a4ff-179d-4f4e-ac78-84b527bea2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.336498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540a4ff-179d-4f4e-ac78-84b527bea2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.356464 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b4ebf-026f-4591-a38d-dab4f66ecf19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f07c8b9747cf48ada26a23f2d13768f6, user_id=d2601c7687184bb49d3476d3faff85a1, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.359388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7540a4ff-179d-4f4e-ac78-84b527bea2c4 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:12.360407 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b4ebf-026f-4591-a38d-dab4f66ecf19 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.360871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7540a4ff-179d-4f4e-ac78-84b527bea2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:12.375783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1208/2426] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:12.376935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540a4ff-179d-4f4e-ac78-84b527bea2c4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.385754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1219/2427] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:12 2022] PUT /identity/v3/projects/f07c8b9747cf48ada26a23f2d13768f6/users/d2601c7687184bb49d3476d3faff85a1/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:12.385754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b4ebf-026f-4591-a38d-dab4f66ecf19 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.389544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540a4ff-179d-4f4e-ac78-84b527bea2c4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.459048 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30825f6-4844-477c-81b2-e35a6a5f4033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.460085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30825f6-4844-477c-81b2-e35a6a5f4033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.460512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30825f6-4844-477c-81b2-e35a6a5f4033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.460900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30825f6-4844-477c-81b2-e35a6a5f4033 admin admin] PATH_INFO: `/v3/projects/0dc2576ab0fb408cb6b946cc55f575f6/users/a07b74bde01645589898d33a4d5705c2/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.466298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b4ebf-026f-4591-a38d-dab4f66ecf19 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.476709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acaabb-f6e9-40c9-81a5-ea12306f539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.477709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acaabb-f6e9-40c9-81a5-ea12306f539d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.477993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acaabb-f6e9-40c9-81a5-ea12306f539d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.478270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acaabb-f6e9-40c9-81a5-ea12306f539d admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.479041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acaabb-f6e9-40c9-81a5-ea12306f539d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:12.479344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acaabb-f6e9-40c9-81a5-ea12306f539d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.480967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acaabb-f6e9-40c9-81a5-ea12306f539d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.488086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30825f6-4844-477c-81b2-e35a6a5f4033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc2576ab0fb408cb6b946cc55f575f6, user_id=a07b74bde01645589898d33a4d5705c2, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.491039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30825f6-4844-477c-81b2-e35a6a5f4033 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.500737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1220/2428] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:11:12.503652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc5814-04a5-4c3a-9ceb-455e2eca5249 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.504721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc5814-04a5-4c3a-9ceb-455e2eca5249 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.504963 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc5814-04a5-4c3a-9ceb-455e2eca5249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.512462 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1209/2429] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:12 2022] PUT /identity/v3/projects/0dc2576ab0fb408cb6b946cc55f575f6/users/a07b74bde01645589898d33a4d5705c2/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:12.514734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30825f6-4844-477c-81b2-e35a6a5f4033 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.538590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e9dc5814-04a5-4c3a-9ceb-455e2eca5249 None None] MFA Rules not processed for user `d2601c7687184bb49d3476d3faff85a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:12.540042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e9dc5814-04a5-4c3a-9ceb-455e2eca5249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:12.597387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30825f6-4844-477c-81b2-e35a6a5f4033 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.604572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d134c-1969-43cf-9a08-5ff95a8fa6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.605402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d134c-1969-43cf-9a08-5ff95a8fa6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.605755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d134c-1969-43cf-9a08-5ff95a8fa6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.606176 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d134c-1969-43cf-9a08-5ff95a8fa6f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.607077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d134c-1969-43cf-9a08-5ff95a8fa6f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.609495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d134c-1969-43cf-9a08-5ff95a8fa6f4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.642146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1221/2430] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:12.642882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc5814-04a5-4c3a-9ceb-455e2eca5249 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.655212 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1210/2431] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:12 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:12.655212 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d134c-1969-43cf-9a08-5ff95a8fa6f4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.659114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc5814-04a5-4c3a-9ceb-455e2eca5249 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.660653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:12.720851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:12.735687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7566f3-0949-407a-bf9d-2d5a2b842aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.736355 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7566f3-0949-407a-bf9d-2d5a2b842aab admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.736681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7566f3-0949-407a-bf9d-2d5a2b842aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.737070 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7566f3-0949-407a-bf9d-2d5a2b842aab admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.737999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7566f3-0949-407a-bf9d-2d5a2b842aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.740315 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7566f3-0949-407a-bf9d-2d5a2b842aab admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.753447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d134c-1969-43cf-9a08-5ff95a8fa6f4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.763424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1222/2432] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:12.763424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7566f3-0949-407a-bf9d-2d5a2b842aab admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.764894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084dd76-6d93-497b-a845-9eb16691228b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.765605 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084dd76-6d93-497b-a845-9eb16691228b admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.765923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084dd76-6d93-497b-a845-9eb16691228b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.766195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084dd76-6d93-497b-a845-9eb16691228b admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.766962 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084dd76-6d93-497b-a845-9eb16691228b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.768986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084dd76-6d93-497b-a845-9eb16691228b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.777925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7566f3-0949-407a-bf9d-2d5a2b842aab admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.783994 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f774d-ce23-4fc0-8bb9-9858c35e3f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.784753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f774d-ce23-4fc0-8bb9-9858c35e3f78 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.784985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f774d-ce23-4fc0-8bb9-9858c35e3f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.786281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f774d-ce23-4fc0-8bb9-9858c35e3f78 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.786958 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f774d-ce23-4fc0-8bb9-9858c35e3f78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.789089 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f774d-ce23-4fc0-8bb9-9858c35e3f78 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.815735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1211/2433] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:12 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:12.817189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084dd76-6d93-497b-a845-9eb16691228b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.821684 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1223/2434] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:12 2022] POST /identity/v3/users => generated 287 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:12.821955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f774d-ce23-4fc0-8bb9-9858c35e3f78 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.833536 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f774d-ce23-4fc0-8bb9-9858c35e3f78 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.928043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084dd76-6d93-497b-a845-9eb16691228b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:12.932721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5c1fe-071f-48fd-8da8-724b60908276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.933491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5c1fe-071f-48fd-8da8-724b60908276 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.933875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5c1fe-071f-48fd-8da8-724b60908276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.934013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b6ee1-ece4-4e81-9c15-9b5aa8c56640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:12.934280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5c1fe-071f-48fd-8da8-724b60908276 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.935047 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5c1fe-071f-48fd-8da8-724b60908276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.935161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b6ee1-ece4-4e81-9c15-9b5aa8c56640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.935442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b6ee1-ece4-4e81-9c15-9b5aa8c56640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.935778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b6ee1-ece4-4e81-9c15-9b5aa8c56640 admin admin] PATH_INFO: `/v3/projects/0dc2576ab0fb408cb6b946cc55f575f6/users/a07b74bde01645589898d33a4d5705c2/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:12.937274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5c1fe-071f-48fd-8da8-724b60908276 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.938944 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b6ee1-ece4-4e81-9c15-9b5aa8c56640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc2576ab0fb408cb6b946cc55f575f6, user_id=a07b74bde01645589898d33a4d5705c2, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:12.942487 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b6ee1-ece4-4e81-9c15-9b5aa8c56640 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:12.956382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1212/2435] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:12 2022] PUT /identity/v3/projects/0dc2576ab0fb408cb6b946cc55f575f6/users/a07b74bde01645589898d33a4d5705c2/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:12.957451 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b6ee1-ece4-4e81-9c15-9b5aa8c56640 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:12.971691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1224/2436] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:12 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:12.971691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c00e7-30bf-49ab-b286-081ed7b6a568 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:12.971691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c00e7-30bf-49ab-b286-081ed7b6a568 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:12.972178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c00e7-30bf-49ab-b286-081ed7b6a568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.008367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e10c00e7-30bf-49ab-b286-081ed7b6a568 None None] MFA Rules not processed for user `a07b74bde01645589898d33a4d5705c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:13.010866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e10c00e7-30bf-49ab-b286-081ed7b6a568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:13.053512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b6ee1-ece4-4e81-9c15-9b5aa8c56640 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.060616 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c821847-e80f-469b-aba5-799b8576765b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.061378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c821847-e80f-469b-aba5-799b8576765b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.061696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c821847-e80f-469b-aba5-799b8576765b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.061966 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c821847-e80f-469b-aba5-799b8576765b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.062786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c821847-e80f-469b-aba5-799b8576765b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.064548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c821847-e80f-469b-aba5-799b8576765b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.076882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1213/2437] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:13.078213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c821847-e80f-469b-aba5-799b8576765b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.089663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c821847-e80f-469b-aba5-799b8576765b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.097902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8bce65-95d5-4429-ba51-37318585ff61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.099484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bce65-95d5-4429-ba51-37318585ff61 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.099995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bce65-95d5-4429-ba51-37318585ff61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.100491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bce65-95d5-4429-ba51-37318585ff61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.102000 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8bce65-95d5-4429-ba51-37318585ff61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.108583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8bce65-95d5-4429-ba51-37318585ff61 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.128171 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1225/2438] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:13.136594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c00e7-30bf-49ab-b286-081ed7b6a568 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.139077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1214/2439] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:13.140509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8bce65-95d5-4429-ba51-37318585ff61 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.155731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c00e7-30bf-49ab-b286-081ed7b6a568 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.158777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8bce65-95d5-4429-ba51-37318585ff61 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.161340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a7c03-2941-446b-8f3a-315d8e7315f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.162092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a7c03-2941-446b-8f3a-315d8e7315f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.162381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a7c03-2941-446b-8f3a-315d8e7315f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.162659 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a7c03-2941-446b-8f3a-315d8e7315f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.163400 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a7c03-2941-446b-8f3a-315d8e7315f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.164287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc08fd5-11fa-4204-beef-805184cd90c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.165404 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc08fd5-11fa-4204-beef-805184cd90c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.165743 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc08fd5-11fa-4204-beef-805184cd90c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.165995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc08fd5-11fa-4204-beef-805184cd90c1 admin admin] PATH_INFO: `/v3/projects/7084414cb7f8430392b7c89b4f6a519f/users/073409de354146ab818c2a4c336639ce/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.166161 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a7c03-2941-446b-8f3a-315d8e7315f0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.168793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:13.182371 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1226/2440] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:13.183311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a7c03-2941-446b-8f3a-315d8e7315f0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.187716 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0f3a7c03-2941-446b-8f3a-315d8e7315f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:13.192154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc08fd5-11fa-4204-beef-805184cd90c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7084414cb7f8430392b7c89b4f6a519f, user_id=073409de354146ab818c2a4c336639ce, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.198019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc08fd5-11fa-4204-beef-805184cd90c1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.220105 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1215/2441] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:13 2022] PUT /identity/v3/projects/7084414cb7f8430392b7c89b4f6a519f/users/073409de354146ab818c2a4c336639ce/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:13.220586 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc08fd5-11fa-4204-beef-805184cd90c1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.246977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:13.264797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a7c03-2941-446b-8f3a-315d8e7315f0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.295945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc08fd5-11fa-4204-beef-805184cd90c1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.301081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee90d42-ffff-45a5-abb8-c882d0ca93d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.302014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee90d42-ffff-45a5-abb8-c882d0ca93d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.302325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee90d42-ffff-45a5-abb8-c882d0ca93d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.302608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee90d42-ffff-45a5-abb8-c882d0ca93d0 admin admin] PATH_INFO: `/v3/projects/118eea41e8ef41b58e5e525b0fa93c5f/users/6fccd472a19c4bd3adf9d9dbecde431a/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.324735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee90d42-ffff-45a5-abb8-c882d0ca93d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=118eea41e8ef41b58e5e525b0fa93c5f, user_id=6fccd472a19c4bd3adf9d9dbecde431a, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.326168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6193c28-0b6e-442b-bcf5-ae9294f93453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.327520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee90d42-ffff-45a5-abb8-c882d0ca93d0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.329473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6193c28-0b6e-442b-bcf5-ae9294f93453 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.329788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6193c28-0b6e-442b-bcf5-ae9294f93453 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.330067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6193c28-0b6e-442b-bcf5-ae9294f93453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.345721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1216/2442] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:13 2022] PUT /identity/v3/projects/118eea41e8ef41b58e5e525b0fa93c5f/users/6fccd472a19c4bd3adf9d9dbecde431a/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:13.346395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee90d42-ffff-45a5-abb8-c882d0ca93d0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.406945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6193c28-0b6e-442b-bcf5-ae9294f93453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.408417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6193c28-0b6e-442b-bcf5-ae9294f93453 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.418881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1227/2443] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:13.421496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837549a4-a2c4-4f41-b086-65959c0b0f05 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.421975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837549a4-a2c4-4f41-b086-65959c0b0f05 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.422362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837549a4-a2c4-4f41-b086-65959c0b0f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.428783 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee90d42-ffff-45a5-abb8-c882d0ca93d0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.435337 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef00c5-02b6-42e8-87f1-e718f6c8902e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.436056 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef00c5-02b6-42e8-87f1-e718f6c8902e admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.436335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef00c5-02b6-42e8-87f1-e718f6c8902e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.436608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef00c5-02b6-42e8-87f1-e718f6c8902e admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.437342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef00c5-02b6-42e8-87f1-e718f6c8902e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.439224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef00c5-02b6-42e8-87f1-e718f6c8902e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.450821 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1217/2444] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:13.455181 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef00c5-02b6-42e8-87f1-e718f6c8902e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.463741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-837549a4-a2c4-4f41-b086-65959c0b0f05 None None] MFA Rules not processed for user `6fccd472a19c4bd3adf9d9dbecde431a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:13.465112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-837549a4-a2c4-4f41-b086-65959c0b0f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:13.469232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef00c5-02b6-42e8-87f1-e718f6c8902e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.476174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8167b-7daa-448b-8c99-0cf115a0d5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.476174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8167b-7daa-448b-8c99-0cf115a0d5c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.476174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8167b-7daa-448b-8c99-0cf115a0d5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.476174 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8167b-7daa-448b-8c99-0cf115a0d5c9 admin admin] PATH_INFO: `/v3/projects/7084414cb7f8430392b7c89b4f6a519f/users/073409de354146ab818c2a4c336639ce/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.479103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8167b-7daa-448b-8c99-0cf115a0d5c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7084414cb7f8430392b7c89b4f6a519f, user_id=073409de354146ab818c2a4c336639ce, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.482017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8167b-7daa-448b-8c99-0cf115a0d5c9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.493427 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1218/2445] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:13 2022] PUT /identity/v3/projects/7084414cb7f8430392b7c89b4f6a519f/users/073409de354146ab818c2a4c336639ce/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:13.507891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90619b2e-a2c2-4eef-9dd5-3061ad49d595 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.508407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90619b2e-a2c2-4eef-9dd5-3061ad49d595 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.508871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90619b2e-a2c2-4eef-9dd5-3061ad49d595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.543377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-90619b2e-a2c2-4eef-9dd5-3061ad49d595 None None] MFA Rules not processed for user `073409de354146ab818c2a4c336639ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:13.544759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-90619b2e-a2c2-4eef-9dd5-3061ad49d595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:13.546164 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1228/2446] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:13.546866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837549a4-a2c4-4f41-b086-65959c0b0f05 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.588023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:13.590271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:13.607292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837549a4-a2c4-4f41-b086-65959c0b0f05 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.621760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1219/2447] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:13.622756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90619b2e-a2c2-4eef-9dd5-3061ad49d595 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.670740 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26f524-f9b4-4b24-89bf-e60857a0959f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.671405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26f524-f9b4-4b24-89bf-e60857a0959f service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.671685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26f524-f9b4-4b24-89bf-e60857a0959f service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.671993 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26f524-f9b4-4b24-89bf-e60857a0959f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.673899 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-be26f524-f9b4-4b24-89bf-e60857a0959f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:13.690081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90619b2e-a2c2-4eef-9dd5-3061ad49d595 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.694636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7aa860-8fc9-416d-bd19-85a70555484b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.695296 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7aa860-8fc9-416d-bd19-85a70555484b admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.695577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7aa860-8fc9-416d-bd19-85a70555484b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.696013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7aa860-8fc9-416d-bd19-85a70555484b admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.696837 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7aa860-8fc9-416d-bd19-85a70555484b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.698817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7aa860-8fc9-416d-bd19-85a70555484b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.726056 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1220/2448] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:13 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:13.726876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7aa860-8fc9-416d-bd19-85a70555484b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.728661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4f7aa860-8fc9-416d-bd19-85a70555484b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:13.737314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26f524-f9b4-4b24-89bf-e60857a0959f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.738836 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26f524-f9b4-4b24-89bf-e60857a0959f service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.748337 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1229/2449] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:13.750486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26f524-f9b4-4b24-89bf-e60857a0959f service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.762764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26f524-f9b4-4b24-89bf-e60857a0959f service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.788759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7aa860-8fc9-416d-bd19-85a70555484b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.793477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3880c4-4358-40c2-ad17-0d538c8dbf55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.794522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3880c4-4358-40c2-ad17-0d538c8dbf55 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.795129 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3880c4-4358-40c2-ad17-0d538c8dbf55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.796006 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3880c4-4358-40c2-ad17-0d538c8dbf55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.827569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f442c-e945-42e5-9dec-af98d944decd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.828342 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f442c-e945-42e5-9dec-af98d944decd admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.828691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f442c-e945-42e5-9dec-af98d944decd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.829056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f442c-e945-42e5-9dec-af98d944decd admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.829793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f442c-e945-42e5-9dec-af98d944decd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.831822 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f442c-e945-42e5-9dec-af98d944decd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.857228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1230/2450] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:13 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:13.861061 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f442c-e945-42e5-9dec-af98d944decd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.872115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3880c4-4358-40c2-ad17-0d538c8dbf55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.872956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f442c-e945-42e5-9dec-af98d944decd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.873841 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3880c4-4358-40c2-ad17-0d538c8dbf55 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.879507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55724b-a1ac-4cc9-95e1-8bb2b8e02b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.880749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55724b-a1ac-4cc9-95e1-8bb2b8e02b94 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.880749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55724b-a1ac-4cc9-95e1-8bb2b8e02b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.880987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55724b-a1ac-4cc9-95e1-8bb2b8e02b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.881786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55724b-a1ac-4cc9-95e1-8bb2b8e02b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.884713 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55724b-a1ac-4cc9-95e1-8bb2b8e02b94 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.886541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1221/2451] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:13.900768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1231/2452] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:13.905104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3880c4-4358-40c2-ad17-0d538c8dbf55 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:13.918039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3880c4-4358-40c2-ad17-0d538c8dbf55 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:13.924382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff140a2-b4e3-4fed-9970-0b95c68a6483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:13.925743 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff140a2-b4e3-4fed-9970-0b95c68a6483 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:13.926028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff140a2-b4e3-4fed-9970-0b95c68a6483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:13.926339 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff140a2-b4e3-4fed-9970-0b95c68a6483 admin admin] PATH_INFO: `/v3/projects/55c8142e2062474297b44d53f7e92c55/users/b4d6bddbbcc54b3fbc5ceaef3630b191/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:13.938285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:13.966937 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff140a2-b4e3-4fed-9970-0b95c68a6483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c8142e2062474297b44d53f7e92c55, user_id=b4d6bddbbcc54b3fbc5ceaef3630b191, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:13.970627 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff140a2-b4e3-4fed-9970-0b95c68a6483 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:13.996843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1222/2453] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:13 2022] PUT /identity/v3/projects/55c8142e2062474297b44d53f7e92c55/users/b4d6bddbbcc54b3fbc5ceaef3630b191/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:14.003134 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55724b-a1ac-4cc9-95e1-8bb2b8e02b94 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:14.021074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55724b-a1ac-4cc9-95e1-8bb2b8e02b94 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:14.049461 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:14.126166 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4711150-b305-45ef-a4a4-8d4239a98a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:14.126877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4711150-b305-45ef-a4a4-8d4239a98a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:14.127187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4711150-b305-45ef-a4a4-8d4239a98a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:14.127441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4711150-b305-45ef-a4a4-8d4239a98a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:14.128534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4711150-b305-45ef-a4a4-8d4239a98a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:14.130361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4711150-b305-45ef-a4a4-8d4239a98a1a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:14.155656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1232/2454] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:14.159847 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff140a2-b4e3-4fed-9970-0b95c68a6483 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:14.173233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff140a2-b4e3-4fed-9970-0b95c68a6483 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:14.180819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72996bb2-ef02-455a-bf35-e7de17a047b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:14.181877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72996bb2-ef02-455a-bf35-e7de17a047b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:14.182219 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72996bb2-ef02-455a-bf35-e7de17a047b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:14.182542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72996bb2-ef02-455a-bf35-e7de17a047b6 admin admin] PATH_INFO: `/v3/projects/55c8142e2062474297b44d53f7e92c55/users/b4d6bddbbcc54b3fbc5ceaef3630b191/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:14.184872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72996bb2-ef02-455a-bf35-e7de17a047b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c8142e2062474297b44d53f7e92c55, user_id=b4d6bddbbcc54b3fbc5ceaef3630b191, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:14.187665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72996bb2-ef02-455a-bf35-e7de17a047b6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:14.201588 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1223/2455] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:14 2022] PUT /identity/v3/projects/55c8142e2062474297b44d53f7e92c55/users/b4d6bddbbcc54b3fbc5ceaef3630b191/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:14.213232 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cad8e-8991-4c40-9673-7213b038eee0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:14.213537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cad8e-8991-4c40-9673-7213b038eee0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:14.214296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cad8e-8991-4c40-9673-7213b038eee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:14.243111 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1e8cad8e-8991-4c40-9673-7213b038eee0 None None] MFA Rules not processed for user `b4d6bddbbcc54b3fbc5ceaef3630b191`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:14.244874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1e8cad8e-8991-4c40-9673-7213b038eee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:14.329576 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1233/2456] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:14.341868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72996bb2-ef02-455a-bf35-e7de17a047b6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:14.375324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cad8e-8991-4c40-9673-7213b038eee0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:14.407302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72996bb2-ef02-455a-bf35-e7de17a047b6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:14.411254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6b664-66cf-4c6a-b19b-7cb2ac9382e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:14.411916 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6b664-66cf-4c6a-b19b-7cb2ac9382e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:14.412204 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6b664-66cf-4c6a-b19b-7cb2ac9382e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:14.412483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6b664-66cf-4c6a-b19b-7cb2ac9382e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:14.422609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6b664-66cf-4c6a-b19b-7cb2ac9382e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:14.424084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6b664-66cf-4c6a-b19b-7cb2ac9382e2 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:14.435393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1224/2457] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:14.447035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cad8e-8991-4c40-9673-7213b038eee0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:14.452501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c633c37-ad46-4080-9fcf-f600f2de76b3 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073409de354146ab818c2a4c336639ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7084414cb7f8430392b7c89b4f6a519f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:14.453522 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c633c37-ad46-4080-9fcf-f600f2de76b3 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:14.453911 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c633c37-ad46-4080-9fcf-f600f2de76b3 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:14.454199 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c633c37-ad46-4080-9fcf-f600f2de76b3 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:14.455038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c633c37-ad46-4080-9fcf-f600f2de76b3 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:14.455369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c633c37-ad46-4080-9fcf-f600f2de76b3 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:14.457157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c633c37-ad46-4080-9fcf-f600f2de76b3 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:14.471132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1234/2458] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:11:14.475802 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6b664-66cf-4c6a-b19b-7cb2ac9382e2 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:14.490135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6b664-66cf-4c6a-b19b-7cb2ac9382e2 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:14.497307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1de0a-5d2a-41bb-9b18-8285a0ee2419 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073409de354146ab818c2a4c336639ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7084414cb7f8430392b7c89b4f6a519f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:14.498115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1de0a-5d2a-41bb-9b18-8285a0ee2419 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:14.498523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1de0a-5d2a-41bb-9b18-8285a0ee2419 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:14.498821 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1de0a-5d2a-41bb-9b18-8285a0ee2419 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:14.499569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1de0a-5d2a-41bb-9b18-8285a0ee2419 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:14.502213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1de0a-5d2a-41bb-9b18-8285a0ee2419 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:14.518574 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c633c37-ad46-4080-9fcf-f600f2de76b3 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:14.533351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c633c37-ad46-4080-9fcf-f600f2de76b3 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:14.539285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d709170-f562-4536-86c8-7e3e00d5595e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:14.539735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1225/2459] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:14 2022] POST /identity/v3/projects => generated 354 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:14.542795 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d709170-f562-4536-86c8-7e3e00d5595e service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:14.543092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d709170-f562-4536-86c8-7e3e00d5595e service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:14.543379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d709170-f562-4536-86c8-7e3e00d5595e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:14.543734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1de0a-5d2a-41bb-9b18-8285a0ee2419 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:14.589048 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:14.623377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d709170-f562-4536-86c8-7e3e00d5595e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:14.625184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d709170-f562-4536-86c8-7e3e00d5595e service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:14.625785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1de0a-5d2a-41bb-9b18-8285a0ee2419 tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:14.631507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5710e-b1fb-4566-9e76-939b87049ece tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073409de354146ab818c2a4c336639ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7084414cb7f8430392b7c89b4f6a519f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:14.632454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5710e-b1fb-4566-9e76-939b87049ece tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:14.632682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5710e-b1fb-4566-9e76-939b87049ece tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:14.632982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5710e-b1fb-4566-9e76-939b87049ece tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:14.634685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5710e-b1fb-4566-9e76-939b87049ece tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:14.635100 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5710e-b1fb-4566-9e76-939b87049ece tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:14.636599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1235/2460] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:14.637150 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5710e-b1fb-4566-9e76-939b87049ece tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:14.652045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1226/2461] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:11:14.694914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d709170-f562-4536-86c8-7e3e00d5595e service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:14.706572 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d709170-f562-4536-86c8-7e3e00d5595e service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:14.711702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44363a54-0000-4f5a-b5d2-e723f2ac916e tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073409de354146ab818c2a4c336639ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7084414cb7f8430392b7c89b4f6a519f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:14.712737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44363a54-0000-4f5a-b5d2-e723f2ac916e tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:14.713014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44363a54-0000-4f5a-b5d2-e723f2ac916e tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:14.713291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44363a54-0000-4f5a-b5d2-e723f2ac916e tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] PATH_INFO: `/v3/projects/a89c85b8ba9a438c952201e73ab2b212` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:14.714514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44363a54-0000-4f5a-b5d2-e723f2ac916e tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a89c85b8ba9a438c952201e73ab2b212)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:14.716426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44363a54-0000-4f5a-b5d2-e723f2ac916e tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:14.780415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-44363a54-0000-4f5a-b5d2-e723f2ac916e tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89c85b8ba9a438c952201e73ab2b212', 'request_id': 'req-44363a54-0000-4f5a-b5d2-e723f2ac916e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:14.787145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-44363a54-0000-4f5a-b5d2-e723f2ac916e tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89c85b8ba9a438c952201e73ab2b212', 'request_id': 'req-44363a54-0000-4f5a-b5d2-e723f2ac916e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:14.796080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-44363a54-0000-4f5a-b5d2-e723f2ac916e tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89c85b8ba9a438c952201e73ab2b212', 'request_id': 'req-44363a54-0000-4f5a-b5d2-e723f2ac916e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:14.808139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-44363a54-0000-4f5a-b5d2-e723f2ac916e tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] The token cache is being invalidate because project a89c85b8ba9a438c952201e73ab2b212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:14.808536 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-44363a54-0000-4f5a-b5d2-e723f2ac916e tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:14.809437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-44363a54-0000-4f5a-b5d2-e723f2ac916e tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:14.811164 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1236/2462] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:14 2022] DELETE /identity/v3/projects/a89c85b8ba9a438c952201e73ab2b212 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:15.041186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5710e-b1fb-4566-9e76-939b87049ece tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:15.043307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-65f5710e-b1fb-4566-9e76-939b87049ece tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:15.098280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:15.118235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5710e-b1fb-4566-9e76-939b87049ece tempest-VolumeQuotaClassesTest-721842064 tempest-VolumeQuotaClassesTest-721842064-project-admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:15.196148 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:15.197412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:15.197740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:15.198025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] PATH_INFO: `/v3/users/d2601c7687184bb49d3476d3faff85a1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:15.199295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2601c7687184bb49d3476d3faff85a1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:15.201352 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:15.260757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2601c7687184bb49d3476d3faff85a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:15.261514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2601c7687184bb49d3476d3faff85a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:15.262165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2601c7687184bb49d3476d3faff85a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:15.270246 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2601c7687184bb49d3476d3faff85a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:15.286079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2601c7687184bb49d3476d3faff85a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:15.305971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1227/2463] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:15 2022] DELETE /identity/v3/users/d2601c7687184bb49d3476d3faff85a1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:15.310888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22d587-9379-407e-a195-42af0f78ab50 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:15.311177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22d587-9379-407e-a195-42af0f78ab50 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:15.311446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22d587-9379-407e-a195-42af0f78ab50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:15.331800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1d22d587-9379-407e-a195-42af0f78ab50 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:15.333160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1d22d587-9379-407e-a195-42af0f78ab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:15.409943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1237/2464] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:15.421272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:15.422960 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:15.506543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e20384-e441-4716-873d-07a5cd4ba250 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:15.511017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d37c0f-7606-4bf8-b47a-e26bf79b7ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:15.511713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d37c0f-7606-4bf8-b47a-e26bf79b7ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:15.511993 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d37c0f-7606-4bf8-b47a-e26bf79b7ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:15.512270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d37c0f-7606-4bf8-b47a-e26bf79b7ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:15.522927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d37c0f-7606-4bf8-b47a-e26bf79b7ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:15.524575 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d37c0f-7606-4bf8-b47a-e26bf79b7ed5 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:15.540057 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1228/2465] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:15.860391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22d587-9379-407e-a195-42af0f78ab50 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:15.862134 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1d22d587-9379-407e-a195-42af0f78ab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:15.880410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22d587-9379-407e-a195-42af0f78ab50 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:15.889290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:15.890504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:15.890982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:15.891297 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] PATH_INFO: `/v3/projects/f07c8b9747cf48ada26a23f2d13768f6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:15.893060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f07c8b9747cf48ada26a23f2d13768f6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:15.895773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:15.962632 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f07c8b9747cf48ada26a23f2d13768f6', 'request_id': 'req-5ea55215-0600-4a25-a535-18cbafeb2683'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:15.968119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:15.970098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f07c8b9747cf48ada26a23f2d13768f6', 'request_id': 'req-5ea55215-0600-4a25-a535-18cbafeb2683'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:15.983679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f07c8b9747cf48ada26a23f2d13768f6', 'request_id': 'req-5ea55215-0600-4a25-a535-18cbafeb2683'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.008032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] The token cache is being invalidate because project f07c8b9747cf48ada26a23f2d13768f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:16.008854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.009946 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.012780 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1238/2466] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:15 2022] DELETE /identity/v3/projects/f07c8b9747cf48ada26a23f2d13768f6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:16.019137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d37c0f-7606-4bf8-b47a-e26bf79b7ed5 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:16.029524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d37c0f-7606-4bf8-b47a-e26bf79b7ed5 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:16.044247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:16.116084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:16.117238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:16.117533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:16.117984 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] PATH_INFO: `/v3/users/073409de354146ab818c2a4c336639ce` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:16.119560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=073409de354146ab818c2a4c336639ce)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:16.121817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:16.186345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073409de354146ab818c2a4c336639ce'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.187425 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073409de354146ab818c2a4c336639ce'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.188230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073409de354146ab818c2a4c336639ce'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.198222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073409de354146ab818c2a4c336639ce'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.214200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073409de354146ab818c2a4c336639ce'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.231709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1229/2467] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:16 2022] DELETE /identity/v3/users/073409de354146ab818c2a4c336639ce => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:16.349193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:16.351375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:16.373430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea55215-0600-4a25-a535-18cbafeb2683 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:16.473064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:16.474029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:16.474306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:16.474607 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] PATH_INFO: `/v3/users/7c1e3814919144b6af526b1c0b3890a8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:16.476113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c1e3814919144b6af526b1c0b3890a8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:16.478492 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:16.535954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:16.539003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:16.560039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0644e7-4b00-4ac8-b6b9-3f50dd0f2175 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:16.571919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c1e3814919144b6af526b1c0b3890a8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.573236 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309547c9-de34-4cb9-a450-326a33f0b014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:16.574240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309547c9-de34-4cb9-a450-326a33f0b014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:16.574531 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309547c9-de34-4cb9-a450-326a33f0b014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:16.574879 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309547c9-de34-4cb9-a450-326a33f0b014 admin admin] PATH_INFO: `/v3/users/80de7c9f3ca64664b1c9920a6eae5222` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:16.576185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309547c9-de34-4cb9-a450-326a33f0b014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80de7c9f3ca64664b1c9920a6eae5222)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:16.578196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309547c9-de34-4cb9-a450-326a33f0b014 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:16.586262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c1e3814919144b6af526b1c0b3890a8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.595538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c1e3814919144b6af526b1c0b3890a8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.596370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c1e3814919144b6af526b1c0b3890a8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.613401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c1e3814919144b6af526b1c0b3890a8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.621114 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1239/2468] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:16 2022] DELETE /identity/v3/users/7c1e3814919144b6af526b1c0b3890a8 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:16.622098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:16.623522 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:16.667578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-309547c9-de34-4cb9-a450-326a33f0b014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80de7c9f3ca64664b1c9920a6eae5222'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.668359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-309547c9-de34-4cb9-a450-326a33f0b014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80de7c9f3ca64664b1c9920a6eae5222'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.669291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-309547c9-de34-4cb9-a450-326a33f0b014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80de7c9f3ca64664b1c9920a6eae5222'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.677577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-309547c9-de34-4cb9-a450-326a33f0b014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80de7c9f3ca64664b1c9920a6eae5222'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.693969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-309547c9-de34-4cb9-a450-326a33f0b014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80de7c9f3ca64664b1c9920a6eae5222'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.712366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1230/2469] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:16 2022] DELETE /identity/v3/users/80de7c9f3ca64664b1c9920a6eae5222 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:16.713853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b612508-6ad5-4bc4-9b5c-8e03bcd81b48 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:16.714141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b612508-6ad5-4bc4-9b5c-8e03bcd81b48 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:16.714437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b612508-6ad5-4bc4-9b5c-8e03bcd81b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:16.723586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2335eaf-e5e8-4169-8704-d8bc27b43f47 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:16.728414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:16.735848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8b612508-6ad5-4bc4-9b5c-8e03bcd81b48 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:16.736442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:16.738276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8b612508-6ad5-4bc4-9b5c-8e03bcd81b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:16.812132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:16.813047 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:16.813326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:16.813613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] PATH_INFO: `/v3/projects/7084414cb7f8430392b7c89b4f6a519f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:16.815011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7084414cb7f8430392b7c89b4f6a519f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:16.817081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:16.819493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1231/2470] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:16.821066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273a42b-df11-4592-a574-0ec96701acc4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:16.821357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273a42b-df11-4592-a574-0ec96701acc4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:16.821689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273a42b-df11-4592-a574-0ec96701acc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:16.845002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a273a42b-df11-4592-a574-0ec96701acc4 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:16.849499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a273a42b-df11-4592-a574-0ec96701acc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:16.861917 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1232/2471] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:16.862910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273a42b-df11-4592-a574-0ec96701acc4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:16.864699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a273a42b-df11-4592-a574-0ec96701acc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:16.906184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7084414cb7f8430392b7c89b4f6a519f', 'request_id': 'req-328992b1-adfb-4657-826d-c8cbc7cccab1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.915506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7084414cb7f8430392b7c89b4f6a519f', 'request_id': 'req-328992b1-adfb-4657-826d-c8cbc7cccab1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.927356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7084414cb7f8430392b7c89b4f6a519f', 'request_id': 'req-328992b1-adfb-4657-826d-c8cbc7cccab1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.941875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] The token cache is being invalidate because project 7084414cb7f8430392b7c89b4f6a519f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:16.942225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.942920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:16.944125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273a42b-df11-4592-a574-0ec96701acc4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:16.944561 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1240/2472] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:16 2022] DELETE /identity/v3/projects/7084414cb7f8430392b7c89b4f6a519f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:16.945386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:16.946871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:17.017973 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca22e3-0eb6-42aa-8fb9-3b4a1de486e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:17.018772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca22e3-0eb6-42aa-8fb9-3b4a1de486e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.019056 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca22e3-0eb6-42aa-8fb9-3b4a1de486e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.019327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca22e3-0eb6-42aa-8fb9-3b4a1de486e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.021207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-45ca22e3-0eb6-42aa-8fb9-3b4a1de486e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:17.029913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328992b1-adfb-4657-826d-c8cbc7cccab1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:17.034518 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cceb3c5-a209-464b-8ee5-403a5d641f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:17.035306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cceb3c5-a209-464b-8ee5-403a5d641f10 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.035770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cceb3c5-a209-464b-8ee5-403a5d641f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.036159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cceb3c5-a209-464b-8ee5-403a5d641f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.038433 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5cceb3c5-a209-464b-8ee5-403a5d641f10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:17.109829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca22e3-0eb6-42aa-8fb9-3b4a1de486e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:17.111971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca22e3-0eb6-42aa-8fb9-3b4a1de486e4 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:17.129011 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1233/2473] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:17.130640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:17.131262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8813e49-d42e-4cad-8a8a-a785fca9881d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.131717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8813e49-d42e-4cad-8a8a-a785fca9881d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.132141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8813e49-d42e-4cad-8a8a-a785fca9881d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.132816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cceb3c5-a209-464b-8ee5-403a5d641f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:17.135445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cceb3c5-a209-464b-8ee5-403a5d641f10 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:17.149345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:17.154372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1241/2474] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:17.157154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c8813e49-d42e-4cad-8a8a-a785fca9881d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:17.158443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c8813e49-d42e-4cad-8a8a-a785fca9881d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:17.167441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1234/2475] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:17.175483 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33ad16-a2c4-4475-86b2-0b4c032c9a4f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.175810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33ad16-a2c4-4475-86b2-0b4c032c9a4f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.176037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33ad16-a2c4-4475-86b2-0b4c032c9a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.195971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2c33ad16-a2c4-4475-86b2-0b4c032c9a4f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:17.197648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2c33ad16-a2c4-4475-86b2-0b4c032c9a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:17.206053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1242/2476] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:17.210418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8813e49-d42e-4cad-8a8a-a785fca9881d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:17.222379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8813e49-d42e-4cad-8a8a-a785fca9881d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:17.228540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432e947-19ab-47ae-8bfc-8b277a7be990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:17.229400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432e947-19ab-47ae-8bfc-8b277a7be990 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.229724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432e947-19ab-47ae-8bfc-8b277a7be990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.230010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432e947-19ab-47ae-8bfc-8b277a7be990 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.230751 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432e947-19ab-47ae-8bfc-8b277a7be990 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:17.231061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432e947-19ab-47ae-8bfc-8b277a7be990 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:17.232710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432e947-19ab-47ae-8bfc-8b277a7be990 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:17.247271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1235/2477] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:11:17.253317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33ad16-a2c4-4475-86b2-0b4c032c9a4f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:17.263759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33ad16-a2c4-4475-86b2-0b4c032c9a4f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:17.269995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68fc3e-180d-4f5e-861f-dfa68bde32ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:17.271146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68fc3e-180d-4f5e-861f-dfa68bde32ac admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.271649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68fc3e-180d-4f5e-861f-dfa68bde32ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.272140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68fc3e-180d-4f5e-861f-dfa68bde32ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.273243 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68fc3e-180d-4f5e-861f-dfa68bde32ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:17.275341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68fc3e-180d-4f5e-861f-dfa68bde32ac admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:17.315437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1243/2478] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:17 2022] POST /identity/v3/projects => generated 409 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:17.319962 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432e947-19ab-47ae-8bfc-8b277a7be990 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:17.333859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432e947-19ab-47ae-8bfc-8b277a7be990 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:17.444477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53375c1-ae95-4df1-854b-34abc7953622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:17.445798 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53375c1-ae95-4df1-854b-34abc7953622 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.446912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53375c1-ae95-4df1-854b-34abc7953622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.447536 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53375c1-ae95-4df1-854b-34abc7953622 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.448482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53375c1-ae95-4df1-854b-34abc7953622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:17.452155 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53375c1-ae95-4df1-854b-34abc7953622 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:17.489340 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1236/2479] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:17 2022] POST /identity/v3/users => generated 305 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:17.494002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68fc3e-180d-4f5e-861f-dfa68bde32ac admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:17.508876 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68fc3e-180d-4f5e-861f-dfa68bde32ac admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:17.515727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08135ee-6ef5-4c74-ac96-6ad5229dee67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:17.516710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08135ee-6ef5-4c74-ac96-6ad5229dee67 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.516914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08135ee-6ef5-4c74-ac96-6ad5229dee67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.517230 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08135ee-6ef5-4c74-ac96-6ad5229dee67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.518300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08135ee-6ef5-4c74-ac96-6ad5229dee67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:17.520204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08135ee-6ef5-4c74-ac96-6ad5229dee67 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:17.538727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1244/2480] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:17.546509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53375c1-ae95-4df1-854b-34abc7953622 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:17.561543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53375c1-ae95-4df1-854b-34abc7953622 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:17.569949 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e931a8-c4a9-4201-bf2a-97c0bb2df3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:17.571806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e931a8-c4a9-4201-bf2a-97c0bb2df3ea admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.572075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e931a8-c4a9-4201-bf2a-97c0bb2df3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.572364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e931a8-c4a9-4201-bf2a-97c0bb2df3ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.573457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e931a8-c4a9-4201-bf2a-97c0bb2df3ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:17.575286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e931a8-c4a9-4201-bf2a-97c0bb2df3ea admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:17.591761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1237/2481] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:11:17.596148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08135ee-6ef5-4c74-ac96-6ad5229dee67 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:17.607297 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08135ee-6ef5-4c74-ac96-6ad5229dee67 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:17.612834 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfba85-ade1-4490-b7eb-c559844eb601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:17.614049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfba85-ade1-4490-b7eb-c559844eb601 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.614384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfba85-ade1-4490-b7eb-c559844eb601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.614605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfba85-ade1-4490-b7eb-c559844eb601 admin admin] PATH_INFO: `/v3/projects/bf5d141b6c2249c4a93813a2ba75657a/users/4055bbcb0f1f443197aef6c60f1afdb2/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.626946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e931a8-c4a9-4201-bf2a-97c0bb2df3ea admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:17.629970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-85e931a8-c4a9-4201-bf2a-97c0bb2df3ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:17.643635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfba85-ade1-4490-b7eb-c559844eb601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf5d141b6c2249c4a93813a2ba75657a, user_id=4055bbcb0f1f443197aef6c60f1afdb2, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:17.647306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfba85-ade1-4490-b7eb-c559844eb601 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:17.651452 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e931a8-c4a9-4201-bf2a-97c0bb2df3ea admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:17.659220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:17.660194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.661186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.661530 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c admin admin] PATH_INFO: `/v3/projects/f7a1fb14d292479387cb49af03570e3a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.663412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a1fb14d292479387cb49af03570e3a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:17.667221 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:17.678770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1245/2482] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:17 2022] PUT /identity/v3/projects/bf5d141b6c2249c4a93813a2ba75657a/users/4055bbcb0f1f443197aef6c60f1afdb2/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:17.679667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfba85-ade1-4490-b7eb-c559844eb601 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:17.682211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0cdfba85-ade1-4490-b7eb-c559844eb601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:17.762707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a1fb14d292479387cb49af03570e3a', 'request_id': 'req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:17.768348 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfba85-ade1-4490-b7eb-c559844eb601 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:17.774179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a1fb14d292479387cb49af03570e3a', 'request_id': 'req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:17.783507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a1fb14d292479387cb49af03570e3a', 'request_id': 'req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:17.802805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c admin admin] The token cache is being invalidate because project f7a1fb14d292479387cb49af03570e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:17.803101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:17.804336 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ee44cf1a-8f9e-44c2-b85d-05d8cd095a5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:17.806266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1238/2483] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:17 2022] DELETE /identity/v3/projects/f7a1fb14d292479387cb49af03570e3a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:17.807952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b653433-fa47-440c-a6e9-9b69015c2d61 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.808243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b653433-fa47-440c-a6e9-9b69015c2d61 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.808731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b653433-fa47-440c-a6e9-9b69015c2d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.839650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3b653433-fa47-440c-a6e9-9b69015c2d61 None None] MFA Rules not processed for user `4055bbcb0f1f443197aef6c60f1afdb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:17.840895 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3b653433-fa47-440c-a6e9-9b69015c2d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:17.845422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:17.846377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.846664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.846959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] PATH_INFO: `/v3/projects/32f42b4279cf4029ba2d5f077117a90a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.848265 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=32f42b4279cf4029ba2d5f077117a90a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:17.850314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:17.885709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:17.894820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:17.906169 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32f42b4279cf4029ba2d5f077117a90a', 'request_id': 'req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:17.914140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32f42b4279cf4029ba2d5f077117a90a', 'request_id': 'req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:17.918963 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1239/2484] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:17 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:17.924512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32f42b4279cf4029ba2d5f077117a90a', 'request_id': 'req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:17.924754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7793477-71f6-4166-bdf7-3ba05cc504eb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:17.924980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7793477-71f6-4166-bdf7-3ba05cc504eb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:17.925259 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7793477-71f6-4166-bdf7-3ba05cc504eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:17.938895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] The token cache is being invalidate because project 32f42b4279cf4029ba2d5f077117a90a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:17.939185 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:17.939866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:17.941514 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1246/2485] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:17 2022] DELETE /identity/v3/projects/32f42b4279cf4029ba2d5f077117a90a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:17.944947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:17.944947 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:17.954778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d7793477-71f6-4166-bdf7-3ba05cc504eb None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:17.956153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d7793477-71f6-4166-bdf7-3ba05cc504eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:18.018380 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c276d1-4b9e-49ad-9d8d-0232aa02df9e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.025283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1240/2486] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:18.025989 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cc0ad-1700-46e7-ae69-fb4ebe64dcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.026336 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7793477-71f6-4166-bdf7-3ba05cc504eb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.026948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cc0ad-1700-46e7-ae69-fb4ebe64dcda admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.027442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cc0ad-1700-46e7-ae69-fb4ebe64dcda admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.027936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cc0ad-1700-46e7-ae69-fb4ebe64dcda admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.028386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d7793477-71f6-4166-bdf7-3ba05cc504eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:18.029143 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cc0ad-1700-46e7-ae69-fb4ebe64dcda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.032406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cc0ad-1700-46e7-ae69-fb4ebe64dcda admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.042099 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7793477-71f6-4166-bdf7-3ba05cc504eb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.047123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.048028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.048321 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.048600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] PATH_INFO: `/v3/users/a07b74bde01645589898d33a4d5705c2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.049882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07b74bde01645589898d33a4d5705c2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.052446 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.066437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1247/2487] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:17 2022] POST /identity/v3/projects => generated 409 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:18.066437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1992b5f-8fed-4b80-8f92-a63c19991569 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.066437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1992b5f-8fed-4b80-8f92-a63c19991569 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.066437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1992b5f-8fed-4b80-8f92-a63c19991569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.085511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b1992b5f-8fed-4b80-8f92-a63c19991569 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:18.086887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b1992b5f-8fed-4b80-8f92-a63c19991569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:18.115623 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07b74bde01645589898d33a4d5705c2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:18.116263 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07b74bde01645589898d33a4d5705c2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:18.117073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07b74bde01645589898d33a4d5705c2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:18.128936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07b74bde01645589898d33a4d5705c2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:18.146006 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07b74bde01645589898d33a4d5705c2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:18.163900 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1241/2488] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:18 2022] DELETE /identity/v3/users/a07b74bde01645589898d33a4d5705c2 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:18.164837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1248/2489] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:18.165282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.166937 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:18.169360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1992b5f-8fed-4b80-8f92-a63c19991569 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.180638 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1992b5f-8fed-4b80-8f92-a63c19991569 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.234706 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:18.254080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243ce88-4461-4b0c-97c4-7dc8a5405a6a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.261397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e280d-f96c-4e11-b464-6b957f808501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.262370 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:18.262491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e280d-f96c-4e11-b464-6b957f808501 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.262843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e280d-f96c-4e11-b464-6b957f808501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.263240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e280d-f96c-4e11-b464-6b957f808501 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.264074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e280d-f96c-4e11-b464-6b957f808501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.266224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e280d-f96c-4e11-b464-6b957f808501 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.281364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b5733-63d9-465b-9a27-e020978a50f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.282776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b5733-63d9-465b-9a27-e020978a50f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.283314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b5733-63d9-465b-9a27-e020978a50f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.284423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b5733-63d9-465b-9a27-e020978a50f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.285535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b5733-63d9-465b-9a27-e020978a50f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:18.286100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b5733-63d9-465b-9a27-e020978a50f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.288038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b5733-63d9-465b-9a27-e020978a50f5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.298733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1242/2490] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:18 2022] POST /identity/v3/users => generated 311 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:18.301487 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e280d-f96c-4e11-b464-6b957f808501 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.306700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1249/2491] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:11:18.311159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b5733-63d9-465b-9a27-e020978a50f5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.315267 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e280d-f96c-4e11-b464-6b957f808501 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.321514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e2710-5e8f-401c-8af8-63eb629d0a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.322193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b5733-63d9-465b-9a27-e020978a50f5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.322333 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e2710-5e8f-401c-8af8-63eb629d0a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.322518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e2710-5e8f-401c-8af8-63eb629d0a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.322795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e2710-5e8f-401c-8af8-63eb629d0a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.323557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e2710-5e8f-401c-8af8-63eb629d0a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.325275 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e2710-5e8f-401c-8af8-63eb629d0a32 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.328462 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf68c2-a4c9-47ef-86ad-71edc718a4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.329363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf68c2-a4c9-47ef-86ad-71edc718a4f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.329940 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf68c2-a4c9-47ef-86ad-71edc718a4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.330459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf68c2-a4c9-47ef-86ad-71edc718a4f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.331247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf68c2-a4c9-47ef-86ad-71edc718a4f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.333438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf68c2-a4c9-47ef-86ad-71edc718a4f0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.336731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1243/2492] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:18.340816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e2710-5e8f-401c-8af8-63eb629d0a32 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.353604 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e2710-5e8f-401c-8af8-63eb629d0a32 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.358394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956e1c7-6360-4bad-820f-48509b1be900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.359420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956e1c7-6360-4bad-820f-48509b1be900 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.359641 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956e1c7-6360-4bad-820f-48509b1be900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.359915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956e1c7-6360-4bad-820f-48509b1be900 admin admin] PATH_INFO: `/v3/projects/b23e33fed7be4995b73d3f94222e67fc/users/57c01238c33144c082909aaf92939ed1/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.372567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1250/2493] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:18 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:18.374782 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf68c2-a4c9-47ef-86ad-71edc718a4f0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.378836 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956e1c7-6360-4bad-820f-48509b1be900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23e33fed7be4995b73d3f94222e67fc, user_id=57c01238c33144c082909aaf92939ed1, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.381642 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956e1c7-6360-4bad-820f-48509b1be900 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.406325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1244/2494] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:18 2022] PUT /identity/v3/projects/b23e33fed7be4995b73d3f94222e67fc/users/57c01238c33144c082909aaf92939ed1/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:18.410132 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956e1c7-6360-4bad-820f-48509b1be900 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.483445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf68c2-a4c9-47ef-86ad-71edc718a4f0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.509038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956e1c7-6360-4bad-820f-48509b1be900 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.515255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761771f-998f-4698-9d62-2432b3de4ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.515255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761771f-998f-4698-9d62-2432b3de4ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.515255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761771f-998f-4698-9d62-2432b3de4ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.515750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761771f-998f-4698-9d62-2432b3de4ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.516178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761771f-998f-4698-9d62-2432b3de4ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.518439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761771f-998f-4698-9d62-2432b3de4ae1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.529777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1245/2495] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:18.537595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761771f-998f-4698-9d62-2432b3de4ae1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.559469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761771f-998f-4698-9d62-2432b3de4ae1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.567264 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0a8bc-03ed-432c-a36f-6fde1ad2dc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.568713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0a8bc-03ed-432c-a36f-6fde1ad2dc6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.568910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0a8bc-03ed-432c-a36f-6fde1ad2dc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.569158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0a8bc-03ed-432c-a36f-6fde1ad2dc6e admin admin] PATH_INFO: `/v3/projects/b23e33fed7be4995b73d3f94222e67fc/users/57c01238c33144c082909aaf92939ed1/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.571551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0a8bc-03ed-432c-a36f-6fde1ad2dc6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23e33fed7be4995b73d3f94222e67fc, user_id=57c01238c33144c082909aaf92939ed1, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.574985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0a8bc-03ed-432c-a36f-6fde1ad2dc6e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.585314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56a338-9020-4d9c-a1aa-912175538e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.586257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a338-9020-4d9c-a1aa-912175538e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.586825 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a338-9020-4d9c-a1aa-912175538e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.588900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a338-9020-4d9c-a1aa-912175538e81 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.589078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1246/2496] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:18 2022] PUT /identity/v3/projects/b23e33fed7be4995b73d3f94222e67fc/users/57c01238c33144c082909aaf92939ed1/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:18.589247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56a338-9020-4d9c-a1aa-912175538e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.589415 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0a8bc-03ed-432c-a36f-6fde1ad2dc6e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.590447 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56a338-9020-4d9c-a1aa-912175538e81 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.598020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c6f0a8bc-03ed-432c-a36f-6fde1ad2dc6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:18.619709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1251/2497] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:18 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:18.623701 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a5b47-6afc-4431-bc31-fdc7bbcff00e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.624142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a5b47-6afc-4431-bc31-fdc7bbcff00e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.624569 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a5b47-6afc-4431-bc31-fdc7bbcff00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.665257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6c6a5b47-6afc-4431-bc31-fdc7bbcff00e None None] MFA Rules not processed for user `57c01238c33144c082909aaf92939ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:18.666562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6c6a5b47-6afc-4431-bc31-fdc7bbcff00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:18.679264 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0a8bc-03ed-432c-a36f-6fde1ad2dc6e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.684847 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.685769 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.686051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.686317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] PATH_INFO: `/v3/projects/0dc2576ab0fb408cb6b946cc55f575f6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.687598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc2576ab0fb408cb6b946cc55f575f6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.689584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.723268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:18.736691 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:18.757394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc2576ab0fb408cb6b946cc55f575f6', 'request_id': 'req-3d95b725-d50c-4079-80d6-adb03b2f3f87'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:18.764320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1252/2498] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:18 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:18.764320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a5b47-6afc-4431-bc31-fdc7bbcff00e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.764652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc2576ab0fb408cb6b946cc55f575f6', 'request_id': 'req-3d95b725-d50c-4079-80d6-adb03b2f3f87'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:18.771592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc2576ab0fb408cb6b946cc55f575f6', 'request_id': 'req-3d95b725-d50c-4079-80d6-adb03b2f3f87'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:18.778685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a5b47-6afc-4431-bc31-fdc7bbcff00e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.797744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] The token cache is being invalidate because project 0dc2576ab0fb408cb6b946cc55f575f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:18.798059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:18.798870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:18.801691 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1247/2499] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:18 2022] DELETE /identity/v3/projects/0dc2576ab0fb408cb6b946cc55f575f6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:18.802594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.804499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:18.872865 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95b725-d50c-4079-80d6-adb03b2f3f87 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.873235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa060c0-08aa-4b11-8ec2-ef3a29162bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.873730 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa060c0-08aa-4b11-8ec2-ef3a29162bce admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.874014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa060c0-08aa-4b11-8ec2-ef3a29162bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.874275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa060c0-08aa-4b11-8ec2-ef3a29162bce admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.875016 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa060c0-08aa-4b11-8ec2-ef3a29162bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.876650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa060c0-08aa-4b11-8ec2-ef3a29162bce admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.877092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a8c60-eb92-43d5-879d-afd375ee50b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.878152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a8c60-eb92-43d5-879d-afd375ee50b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.878436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a8c60-eb92-43d5-879d-afd375ee50b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.878717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a8c60-eb92-43d5-879d-afd375ee50b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.880793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e79a8c60-eb92-43d5-879d-afd375ee50b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:18.889259 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1253/2500] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:18.890718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a888988-d4d0-40f4-8b7a-f89542e6bd1c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.890999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a888988-d4d0-40f4-8b7a-f89542e6bd1c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.891506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a888988-d4d0-40f4-8b7a-f89542e6bd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.913681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4a888988-d4d0-40f4-8b7a-f89542e6bd1c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:18.915008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4a888988-d4d0-40f4-8b7a-f89542e6bd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:18.923436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1254/2501] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:18.924336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a888988-d4d0-40f4-8b7a-f89542e6bd1c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.925649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4a888988-d4d0-40f4-8b7a-f89542e6bd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:18.939786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a888988-d4d0-40f4-8b7a-f89542e6bd1c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:18.945211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108aa215-7c10-47aa-b389-417409d59d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:18.946044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108aa215-7c10-47aa-b389-417409d59d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.946639 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108aa215-7c10-47aa-b389-417409d59d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.946910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108aa215-7c10-47aa-b389-417409d59d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.947688 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108aa215-7c10-47aa-b389-417409d59d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.949334 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108aa215-7c10-47aa-b389-417409d59d13 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.963783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1255/2502] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:18.965497 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664079fe-2949-4e4e-bc40-eadb8ec56373 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:18.965771 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664079fe-2949-4e4e-bc40-eadb8ec56373 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:18.965948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664079fe-2949-4e4e-bc40-eadb8ec56373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:18.970268 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a8c60-eb92-43d5-879d-afd375ee50b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:18.971859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a8c60-eb92-43d5-879d-afd375ee50b4 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:18.983579 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1248/2503] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:18 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:18.988218 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-664079fe-2949-4e4e-bc40-eadb8ec56373 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:18.989444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-664079fe-2949-4e4e-bc40-eadb8ec56373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:18.990866 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a8c60-eb92-43d5-879d-afd375ee50b4 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:18.998470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1256/2504] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:19.002481 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664079fe-2949-4e4e-bc40-eadb8ec56373 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:19.005511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a8c60-eb92-43d5-879d-afd375ee50b4 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:19.012949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664079fe-2949-4e4e-bc40-eadb8ec56373 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:19.013905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7d0b6-65dc-4141-85b8-952b0df30b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:19.015303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7d0b6-65dc-4141-85b8-952b0df30b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.015786 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7d0b6-65dc-4141-85b8-952b0df30b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.016252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7d0b6-65dc-4141-85b8-952b0df30b5a admin admin] PATH_INFO: `/v3/projects/f9a0deda992f4b0baf070be5563587c1/users/0801fb7efa4642c180c0006a1970b654/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.018525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15093b4d-e063-4a34-a434-95bd059bbc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:19.019098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15093b4d-e063-4a34-a434-95bd059bbc4d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.019902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15093b4d-e063-4a34-a434-95bd059bbc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.019902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15093b4d-e063-4a34-a434-95bd059bbc4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.020571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15093b4d-e063-4a34-a434-95bd059bbc4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:19.020885 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15093b4d-e063-4a34-a434-95bd059bbc4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:19.022586 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15093b4d-e063-4a34-a434-95bd059bbc4d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:19.037224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1257/2505] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:11:19.041335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7d0b6-65dc-4141-85b8-952b0df30b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a0deda992f4b0baf070be5563587c1, user_id=0801fb7efa4642c180c0006a1970b654, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:19.042652 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15093b4d-e063-4a34-a434-95bd059bbc4d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:19.046937 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7d0b6-65dc-4141-85b8-952b0df30b5a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:19.060754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15093b4d-e063-4a34-a434-95bd059bbc4d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:19.069918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1249/2506] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:18 2022] PUT /identity/v3/projects/f9a0deda992f4b0baf070be5563587c1/users/0801fb7efa4642c180c0006a1970b654/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:19.082524 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9d9f9-631a-47ed-b26a-981067bbc00b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.083160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9d9f9-631a-47ed-b26a-981067bbc00b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.083739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9d9f9-631a-47ed-b26a-981067bbc00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.152473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7eb9d9f9-631a-47ed-b26a-981067bbc00b None None] MFA Rules not processed for user `0801fb7efa4642c180c0006a1970b654`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:19.154652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7eb9d9f9-631a-47ed-b26a-981067bbc00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:19.172761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff630f2d-b454-4627-b334-9fae8fde6ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:19.175945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff630f2d-b454-4627-b334-9fae8fde6ffd admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.176479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff630f2d-b454-4627-b334-9fae8fde6ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.176839 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff630f2d-b454-4627-b334-9fae8fde6ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.178600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff630f2d-b454-4627-b334-9fae8fde6ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:19.181142 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff630f2d-b454-4627-b334-9fae8fde6ffd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:19.205472 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:19.215688 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1258/2507] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:19 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:19.216458 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff630f2d-b454-4627-b334-9fae8fde6ffd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:19.250019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1250/2508] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:19.250856 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:19.251194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9d9f9-631a-47ed-b26a-981067bbc00b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:19.264306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9d9f9-631a-47ed-b26a-981067bbc00b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:19.289855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff630f2d-b454-4627-b334-9fae8fde6ffd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:19.294411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eab345-5100-4a87-bd9d-64fd558b9e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:19.295173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eab345-5100-4a87-bd9d-64fd558b9e2d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.295523 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eab345-5100-4a87-bd9d-64fd558b9e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.295881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eab345-5100-4a87-bd9d-64fd558b9e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.297978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-00eab345-5100-4a87-bd9d-64fd558b9e2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:19.339265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a9cf4-41da-4938-a1a3-aa8f66d011d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:19.340183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a9cf4-41da-4938-a1a3-aa8f66d011d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.340600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a9cf4-41da-4938-a1a3-aa8f66d011d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.341077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a9cf4-41da-4938-a1a3-aa8f66d011d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.342128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a9cf4-41da-4938-a1a3-aa8f66d011d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:19.344860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a9cf4-41da-4938-a1a3-aa8f66d011d1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:19.366596 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eab345-5100-4a87-bd9d-64fd558b9e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:19.368438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eab345-5100-4a87-bd9d-64fd558b9e2d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:19.370096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1251/2509] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:19 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:19.371060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a9cf4-41da-4938-a1a3-aa8f66d011d1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:19.372975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-198a9cf4-41da-4938-a1a3-aa8f66d011d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:19.380305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1259/2510] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:19.381284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eab345-5100-4a87-bd9d-64fd558b9e2d service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:19.394886 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eab345-5100-4a87-bd9d-64fd558b9e2d service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:19.401547 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daf5b4-7261-4a54-8ee1-2cf4c465bb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:19.403094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daf5b4-7261-4a54-8ee1-2cf4c465bb6c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.403094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daf5b4-7261-4a54-8ee1-2cf4c465bb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.403094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daf5b4-7261-4a54-8ee1-2cf4c465bb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.403468 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9daf5b4-7261-4a54-8ee1-2cf4c465bb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:19.404959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9daf5b4-7261-4a54-8ee1-2cf4c465bb6c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:19.421399 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1260/2511] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:19.422603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daf5b4-7261-4a54-8ee1-2cf4c465bb6c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:19.437393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daf5b4-7261-4a54-8ee1-2cf4c465bb6c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:19.447554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d79fe8-664c-418d-8e7d-5e11c8d57efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:19.448007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d79fe8-664c-418d-8e7d-5e11c8d57efb admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.448512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d79fe8-664c-418d-8e7d-5e11c8d57efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.449025 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d79fe8-664c-418d-8e7d-5e11c8d57efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.450474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d79fe8-664c-418d-8e7d-5e11c8d57efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:19.453791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d79fe8-664c-418d-8e7d-5e11c8d57efb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:19.475361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1261/2512] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:19.479754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d79fe8-664c-418d-8e7d-5e11c8d57efb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:19.482778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a9cf4-41da-4938-a1a3-aa8f66d011d1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:19.489378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a28e62-7f07-4f78-8885-8b661a13f8d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:19.490097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a28e62-7f07-4f78-8885-8b661a13f8d1 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.490386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a28e62-7f07-4f78-8885-8b661a13f8d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.490673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a28e62-7f07-4f78-8885-8b661a13f8d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.490843 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d79fe8-664c-418d-8e7d-5e11c8d57efb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:19.495792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04c481-2cde-431e-971d-42a1aec47d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:19.496723 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04c481-2cde-431e-971d-42a1aec47d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.497005 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04c481-2cde-431e-971d-42a1aec47d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.497286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04c481-2cde-431e-971d-42a1aec47d82 admin admin] PATH_INFO: `/v3/projects/bfb2bf1afead415b9d6ee79ad836197d/users/0224639fe5d24820bdead1bba468ddda/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.516657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04c481-2cde-431e-971d-42a1aec47d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb2bf1afead415b9d6ee79ad836197d, user_id=0224639fe5d24820bdead1bba468ddda, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:19.519769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04c481-2cde-431e-971d-42a1aec47d82 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:19.540854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1262/2513] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:19 2022] PUT /identity/v3/projects/bfb2bf1afead415b9d6ee79ad836197d/users/0224639fe5d24820bdead1bba468ddda/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:19.546641 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458914d9-6a4c-4e6e-b642-c0e5676215c6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.546986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458914d9-6a4c-4e6e-b642-c0e5676215c6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.547508 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458914d9-6a4c-4e6e-b642-c0e5676215c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.564059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a28e62-7f07-4f78-8885-8b661a13f8d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:19.565806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a28e62-7f07-4f78-8885-8b661a13f8d1 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:19.573336 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:19.576152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-458914d9-6a4c-4e6e-b642-c0e5676215c6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:19.577883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-458914d9-6a4c-4e6e-b642-c0e5676215c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:19.579453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1252/2514] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:19.678803 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1263/2515] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:19.688826 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a28e62-7f07-4f78-8885-8b661a13f8d1 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:19.690405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d4a28e62-7f07-4f78-8885-8b661a13f8d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:19.789348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a28e62-7f07-4f78-8885-8b661a13f8d1 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:19.862028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458914d9-6a4c-4e6e-b642-c0e5676215c6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:19.875511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b2816-2d1f-4600-9235-b50454ff87c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:19.876272 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458914d9-6a4c-4e6e-b642-c0e5676215c6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:19.876423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b2816-2d1f-4600-9235-b50454ff87c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.876747 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b2816-2d1f-4600-9235-b50454ff87c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.876968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b2816-2d1f-4600-9235-b50454ff87c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.882193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:19.882914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:19.883212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:19.883727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] PATH_INFO: `/v3/users/0801fb7efa4642c180c0006a1970b654` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:19.885146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0801fb7efa4642c180c0006a1970b654)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:19.887193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:19.889300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b2816-2d1f-4600-9235-b50454ff87c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:19.890848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b2816-2d1f-4600-9235-b50454ff87c4 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:19.901957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1253/2516] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:19.928645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:19.944068 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0801fb7efa4642c180c0006a1970b654'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:19.960713 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0801fb7efa4642c180c0006a1970b654'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:19.974225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0801fb7efa4642c180c0006a1970b654'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:19.975003 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0801fb7efa4642c180c0006a1970b654'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:19.990380 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0801fb7efa4642c180c0006a1970b654'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:19.998695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1264/2517] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:19 2022] DELETE /identity/v3/users/0801fb7efa4642c180c0006a1970b654 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:20.004083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fd686-c522-422d-ad02-90ac8f80d869 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:20.004389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fd686-c522-422d-ad02-90ac8f80d869 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:20.004755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fd686-c522-422d-ad02-90ac8f80d869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:20.025136 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3e2fd686-c522-422d-ad02-90ac8f80d869 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:20.026624 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3e2fd686-c522-422d-ad02-90ac8f80d869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:20.101602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1254/2518] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:20.112801 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:20.122275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:20.210347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734aae6-2690-4915-9b07-35a8b7ccdfd7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:20.214905 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca17c7-5960-4148-bc27-59e724e4abbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:20.215704 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca17c7-5960-4148-bc27-59e724e4abbf service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:20.216098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca17c7-5960-4148-bc27-59e724e4abbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:20.216438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca17c7-5960-4148-bc27-59e724e4abbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:20.236987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca17c7-5960-4148-bc27-59e724e4abbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:20.238828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca17c7-5960-4148-bc27-59e724e4abbf service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:20.251017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1265/2519] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:20.633014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fd686-c522-422d-ad02-90ac8f80d869 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:20.634753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3e2fd686-c522-422d-ad02-90ac8f80d869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:20.650191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fd686-c522-422d-ad02-90ac8f80d869 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:20.660101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f966a-085b-4f8a-8994-531dc63ef45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:20.660101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f966a-085b-4f8a-8994-531dc63ef45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:20.660101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f966a-085b-4f8a-8994-531dc63ef45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:20.660101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f966a-085b-4f8a-8994-531dc63ef45b admin admin] PATH_INFO: `/v3/projects/f9a0deda992f4b0baf070be5563587c1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:20.668151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f966a-085b-4f8a-8994-531dc63ef45b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a0deda992f4b0baf070be5563587c1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:20.668151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f966a-085b-4f8a-8994-531dc63ef45b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:20.733768 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5d6f966a-085b-4f8a-8994-531dc63ef45b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a0deda992f4b0baf070be5563587c1', 'request_id': 'req-5d6f966a-085b-4f8a-8994-531dc63ef45b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:20.742417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:20.745514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5d6f966a-085b-4f8a-8994-531dc63ef45b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a0deda992f4b0baf070be5563587c1', 'request_id': 'req-5d6f966a-085b-4f8a-8994-531dc63ef45b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:20.759509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5d6f966a-085b-4f8a-8994-531dc63ef45b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a0deda992f4b0baf070be5563587c1', 'request_id': 'req-5d6f966a-085b-4f8a-8994-531dc63ef45b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:20.773158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2b3ae-a57e-42ae-94f5-43f11386aa62 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:20.773529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2b3ae-a57e-42ae-94f5-43f11386aa62 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:20.774152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2b3ae-a57e-42ae-94f5-43f11386aa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:20.786659 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5d6f966a-085b-4f8a-8994-531dc63ef45b admin admin] The token cache is being invalidate because project f9a0deda992f4b0baf070be5563587c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:20.787043 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5d6f966a-085b-4f8a-8994-531dc63ef45b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:20.788041 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5d6f966a-085b-4f8a-8994-531dc63ef45b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:20.791344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1255/2520] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:20 2022] DELETE /identity/v3/projects/f9a0deda992f4b0baf070be5563587c1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:20.801482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadfd32-fde0-4349-8249-98a0697c4781 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:20.801482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadfd32-fde0-4349-8249-98a0697c4781 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:20.801482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadfd32-fde0-4349-8249-98a0697c4781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:20.828278 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c5d2b3ae-a57e-42ae-94f5-43f11386aa62 None None] MFA Rules not processed for user `0224639fe5d24820bdead1bba468ddda`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:20.830784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c5d2b3ae-a57e-42ae-94f5-43f11386aa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:20.842964 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5fadfd32-fde0-4349-8249-98a0697c4781 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:20.845270 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5fadfd32-fde0-4349-8249-98a0697c4781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:20.972337 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1256/2521] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:20.977680 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc09595-22f4-46c4-9b20-0b6a67e8302b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:20.977931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1266/2522] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:20.978710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc09595-22f4-46c4-9b20-0b6a67e8302b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:20.978710 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc09595-22f4-46c4-9b20-0b6a67e8302b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:20.981455 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2b3ae-a57e-42ae-94f5-43f11386aa62 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:20.985718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c5d2b3ae-a57e-42ae-94f5-43f11386aa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:21.015297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8cc09595-22f4-46c4-9b20-0b6a67e8302b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:21.017985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8cc09595-22f4-46c4-9b20-0b6a67e8302b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:21.031687 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1257/2523] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:21.037254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc09595-22f4-46c4-9b20-0b6a67e8302b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:21.052130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc09595-22f4-46c4-9b20-0b6a67e8302b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:21.058268 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f6ece-ef62-4574-9fe8-3e5821a458b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:21.059758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f6ece-ef62-4574-9fe8-3e5821a458b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:21.059758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f6ece-ef62-4574-9fe8-3e5821a458b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:21.059758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f6ece-ef62-4574-9fe8-3e5821a458b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:21.060651 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f6ece-ef62-4574-9fe8-3e5821a458b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:21.060972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f6ece-ef62-4574-9fe8-3e5821a458b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:21.062755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f6ece-ef62-4574-9fe8-3e5821a458b3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:21.078517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1258/2524] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:11:21.083172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f6ece-ef62-4574-9fe8-3e5821a458b3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:21.095631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:21.100561 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f6ece-ef62-4574-9fe8-3e5821a458b3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:21.113019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8089c2-9a5a-4e4b-9ec5-d90f597956f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:21.113853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8089c2-9a5a-4e4b-9ec5-d90f597956f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:21.114848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2b3ae-a57e-42ae-94f5-43f11386aa62 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:21.114991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8089c2-9a5a-4e4b-9ec5-d90f597956f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:21.115219 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8089c2-9a5a-4e4b-9ec5-d90f597956f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:21.116184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8089c2-9a5a-4e4b-9ec5-d90f597956f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:21.118371 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8089c2-9a5a-4e4b-9ec5-d90f597956f8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:21.124295 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facf71a-552d-40f2-9d2f-c7821b91b61e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:21.125260 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facf71a-552d-40f2-9d2f-c7821b91b61e service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:21.125541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facf71a-552d-40f2-9d2f-c7821b91b61e service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:21.125866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facf71a-552d-40f2-9d2f-c7821b91b61e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:21.136402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facf71a-552d-40f2-9d2f-c7821b91b61e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:21.138019 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facf71a-552d-40f2-9d2f-c7821b91b61e service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:21.156207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1267/2525] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:21.162478 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1259/2526] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:21 2022] POST /identity/v3/projects => generated 369 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:21.169917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facf71a-552d-40f2-9d2f-c7821b91b61e service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:21.188603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facf71a-552d-40f2-9d2f-c7821b91b61e service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:21.284105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f91c49-efb1-4f3f-945f-984791570fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:21.284105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f91c49-efb1-4f3f-945f-984791570fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:21.284105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f91c49-efb1-4f3f-945f-984791570fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:21.284922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f91c49-efb1-4f3f-945f-984791570fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:21.285092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f91c49-efb1-4f3f-945f-984791570fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:21.286990 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f91c49-efb1-4f3f-945f-984791570fd8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:21.312104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1268/2527] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:21 2022] POST /identity/v3/users => generated 285 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:21.316849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8089c2-9a5a-4e4b-9ec5-d90f597956f8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:21.333525 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8089c2-9a5a-4e4b-9ec5-d90f597956f8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:21.341142 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d40aa-6301-4e49-be0b-c42a82d58ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:21.341683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d40aa-6301-4e49-be0b-c42a82d58ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:21.341983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d40aa-6301-4e49-be0b-c42a82d58ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:21.342914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d40aa-6301-4e49-be0b-c42a82d58ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:21.342914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d40aa-6301-4e49-be0b-c42a82d58ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:21.344552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d40aa-6301-4e49-be0b-c42a82d58ce0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:21.360974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1260/2528] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:11:21.373613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f91c49-efb1-4f3f-945f-984791570fd8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:21.390697 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f91c49-efb1-4f3f-945f-984791570fd8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:21.400857 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f86861-d359-4184-b777-f21aaa64a36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:21.401965 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f86861-d359-4184-b777-f21aaa64a36c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:21.402409 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f86861-d359-4184-b777-f21aaa64a36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:21.402849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f86861-d359-4184-b777-f21aaa64a36c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:21.404059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f86861-d359-4184-b777-f21aaa64a36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:21.407498 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f86861-d359-4184-b777-f21aaa64a36c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:21.430286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1269/2529] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:21.431456 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d40aa-6301-4e49-be0b-c42a82d58ce0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:21.446154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d40aa-6301-4e49-be0b-c42a82d58ce0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:21.454078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce5b1f-af8e-4a35-ac63-6eada63b6c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:21.455729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce5b1f-af8e-4a35-ac63-6eada63b6c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:21.456310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce5b1f-af8e-4a35-ac63-6eada63b6c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:21.457149 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce5b1f-af8e-4a35-ac63-6eada63b6c1c admin admin] PATH_INFO: `/v3/projects/e9e91ab36e7a40b7a70f6f69a8230622/users/23a371dedc924f3a8abff682449e33ea/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:21.488453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce5b1f-af8e-4a35-ac63-6eada63b6c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e91ab36e7a40b7a70f6f69a8230622, user_id=23a371dedc924f3a8abff682449e33ea, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:21.492125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce5b1f-af8e-4a35-ac63-6eada63b6c1c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:21.515374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1261/2530] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:21 2022] PUT /identity/v3/projects/e9e91ab36e7a40b7a70f6f69a8230622/users/23a371dedc924f3a8abff682449e33ea/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:21.534126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c20324-be20-4553-9619-302708c873ab None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:21.534733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c20324-be20-4553-9619-302708c873ab None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:21.535311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c20324-be20-4553-9619-302708c873ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:21.595520 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-15c20324-be20-4553-9619-302708c873ab None None] MFA Rules not processed for user `23a371dedc924f3a8abff682449e33ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:21.598256 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-15c20324-be20-4553-9619-302708c873ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:21.715940 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1270/2531] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:21.721989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce5b1f-af8e-4a35-ac63-6eada63b6c1c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:21.807324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce5b1f-af8e-4a35-ac63-6eada63b6c1c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:21.812676 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc18d0f-e2d1-4b2b-88cc-120aecf003d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:21.813359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18d0f-e2d1-4b2b-88cc-120aecf003d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:21.813662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18d0f-e2d1-4b2b-88cc-120aecf003d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:21.813945 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18d0f-e2d1-4b2b-88cc-120aecf003d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:21.814943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc18d0f-e2d1-4b2b-88cc-120aecf003d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:21.817044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc18d0f-e2d1-4b2b-88cc-120aecf003d3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:21.828770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:21.854435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1262/2532] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:21 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:21.863833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c20324-be20-4553-9619-302708c873ab None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:21.881494 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c20324-be20-4553-9619-302708c873ab None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:21.946598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:21.961054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a865709-8f7b-4124-86fd-60c3d13bb788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:21.963599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a865709-8f7b-4124-86fd-60c3d13bb788 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:21.964321 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a865709-8f7b-4124-86fd-60c3d13bb788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:21.964816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a865709-8f7b-4124-86fd-60c3d13bb788 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:21.966321 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a865709-8f7b-4124-86fd-60c3d13bb788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:21.970233 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a865709-8f7b-4124-86fd-60c3d13bb788 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:22.006313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1271/2533] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:21 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:22.011485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc18d0f-e2d1-4b2b-88cc-120aecf003d3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:22.023894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc18d0f-e2d1-4b2b-88cc-120aecf003d3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:22.030692 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34d0ae-5e69-4010-a84f-4b0ac1453333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:22.031811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34d0ae-5e69-4010-a84f-4b0ac1453333 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:22.032254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34d0ae-5e69-4010-a84f-4b0ac1453333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:22.032714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34d0ae-5e69-4010-a84f-4b0ac1453333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:22.033696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34d0ae-5e69-4010-a84f-4b0ac1453333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:22.036630 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34d0ae-5e69-4010-a84f-4b0ac1453333 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:22.051940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1263/2534] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:22.056363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a865709-8f7b-4124-86fd-60c3d13bb788 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:22.068738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a865709-8f7b-4124-86fd-60c3d13bb788 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:22.074783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff648a3-7d45-4859-83b6-b80fe7db1c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:22.075699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff648a3-7d45-4859-83b6-b80fe7db1c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:22.075976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff648a3-7d45-4859-83b6-b80fe7db1c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:22.076249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff648a3-7d45-4859-83b6-b80fe7db1c53 admin admin] PATH_INFO: `/v3/projects/675c68e578c249d2ab70e53b34cd2a08/users/a21256eb490349398560f19100c58fde/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:22.097803 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff648a3-7d45-4859-83b6-b80fe7db1c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=675c68e578c249d2ab70e53b34cd2a08, user_id=a21256eb490349398560f19100c58fde, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:22.100785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff648a3-7d45-4859-83b6-b80fe7db1c53 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:22.139067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1272/2535] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:22 2022] PUT /identity/v3/projects/675c68e578c249d2ab70e53b34cd2a08/users/a21256eb490349398560f19100c58fde/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:22.143731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34d0ae-5e69-4010-a84f-4b0ac1453333 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:22.159685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34d0ae-5e69-4010-a84f-4b0ac1453333 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:22.232001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c31d69-0ab3-4bba-a67e-8c5318fb5f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:22.233001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c31d69-0ab3-4bba-a67e-8c5318fb5f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:22.233396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c31d69-0ab3-4bba-a67e-8c5318fb5f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:22.233828 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c31d69-0ab3-4bba-a67e-8c5318fb5f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:22.235028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c31d69-0ab3-4bba-a67e-8c5318fb5f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:22.237510 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c31d69-0ab3-4bba-a67e-8c5318fb5f2b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:22.250445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1264/2536] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:22.255538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff648a3-7d45-4859-83b6-b80fe7db1c53 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:22.267920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff648a3-7d45-4859-83b6-b80fe7db1c53 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:22.273057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2278479-6965-46f8-a2ac-5e6cd860e225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:22.274118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2278479-6965-46f8-a2ac-5e6cd860e225 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:22.274738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2278479-6965-46f8-a2ac-5e6cd860e225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:22.274738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2278479-6965-46f8-a2ac-5e6cd860e225 admin admin] PATH_INFO: `/v3/projects/675c68e578c249d2ab70e53b34cd2a08/users/a21256eb490349398560f19100c58fde/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:22.276814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2278479-6965-46f8-a2ac-5e6cd860e225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=675c68e578c249d2ab70e53b34cd2a08, user_id=a21256eb490349398560f19100c58fde, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:22.279545 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2278479-6965-46f8-a2ac-5e6cd860e225 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:22.291243 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1273/2537] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:22 2022] PUT /identity/v3/projects/675c68e578c249d2ab70e53b34cd2a08/users/a21256eb490349398560f19100c58fde/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:22.303291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068af54-c060-49a7-b474-18ff21d394a9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:22.303584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068af54-c060-49a7-b474-18ff21d394a9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:22.303867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068af54-c060-49a7-b474-18ff21d394a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:22.333869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4068af54-c060-49a7-b474-18ff21d394a9 None None] MFA Rules not processed for user `a21256eb490349398560f19100c58fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:22.335123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4068af54-c060-49a7-b474-18ff21d394a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:22.421450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1265/2538] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:22 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:22.434030 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2278479-6965-46f8-a2ac-5e6cd860e225 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:22.501495 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2278479-6965-46f8-a2ac-5e6cd860e225 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:22.506209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec149de5-c4da-4082-b273-6305e05a350c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:22.506892 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec149de5-c4da-4082-b273-6305e05a350c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:22.507178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec149de5-c4da-4082-b273-6305e05a350c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:22.507455 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec149de5-c4da-4082-b273-6305e05a350c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:22.519262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec149de5-c4da-4082-b273-6305e05a350c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:22.520945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec149de5-c4da-4082-b273-6305e05a350c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:22.533559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1274/2539] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:22 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:22.584114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068af54-c060-49a7-b474-18ff21d394a9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:22.585746 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4068af54-c060-49a7-b474-18ff21d394a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:22.602960 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068af54-c060-49a7-b474-18ff21d394a9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:22.607908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045ea29-45ab-4f62-a311-3d1cbff0b5f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:22.608650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045ea29-45ab-4f62-a311-3d1cbff0b5f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:22.608972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045ea29-45ab-4f62-a311-3d1cbff0b5f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:22.609278 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045ea29-45ab-4f62-a311-3d1cbff0b5f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:22.678932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045ea29-45ab-4f62-a311-3d1cbff0b5f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:22.680450 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045ea29-45ab-4f62-a311-3d1cbff0b5f5 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:22.685864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:22.693207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1266/2540] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:22 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:22.726844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ac6c6-1efb-4a14-b974-b19aee17d15a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:22.727543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ac6c6-1efb-4a14-b974-b19aee17d15a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:22.727543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ac6c6-1efb-4a14-b974-b19aee17d15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:22.751041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-394ac6c6-1efb-4a14-b974-b19aee17d15a None None] MFA Rules not processed for user `0224639fe5d24820bdead1bba468ddda`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:22.753307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-394ac6c6-1efb-4a14-b974-b19aee17d15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:22.836225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1275/2541] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:26.450141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045ea29-45ab-4f62-a311-3d1cbff0b5f5 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:26.546226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045ea29-45ab-4f62-a311-3d1cbff0b5f5 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:26.551527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:26.552546 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:26.552940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:26.553237 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] PATH_INFO: `/v3/users/23a371dedc924f3a8abff682449e33ea` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:26.554511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a371dedc924f3a8abff682449e33ea)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:26.557201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:26.663941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a371dedc924f3a8abff682449e33ea'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:26.666346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a371dedc924f3a8abff682449e33ea'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:26.668008 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a371dedc924f3a8abff682449e33ea'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:26.701418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a371dedc924f3a8abff682449e33ea'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:26.739031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a371dedc924f3a8abff682449e33ea'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:26.758845 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1267/2542] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:26 2022] DELETE /identity/v3/users/23a371dedc924f3a8abff682449e33ea => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:26.764124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf60354-4ae8-4e38-929d-8b0f081f3c02 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:26.765944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf60354-4ae8-4e38-929d-8b0f081f3c02 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:26.768089 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf60354-4ae8-4e38-929d-8b0f081f3c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:26.798574 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-baf60354-4ae8-4e38-929d-8b0f081f3c02 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:26.800424 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-baf60354-4ae8-4e38-929d-8b0f081f3c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:26.914129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:26.945378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1276/2543] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:26.956958 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:26.958578 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:27.077405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a0a4b-4b52-4a93-b7d1-a84a5fa669ef admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:27.082376 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca06705-88bd-4c46-b861-1d75c8b198c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:27.083433 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca06705-88bd-4c46-b861-1d75c8b198c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:27.085781 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca06705-88bd-4c46-b861-1d75c8b198c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:27.085781 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca06705-88bd-4c46-b861-1d75c8b198c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:27.104761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca06705-88bd-4c46-b861-1d75c8b198c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:27.106726 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca06705-88bd-4c46-b861-1d75c8b198c3 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:27.122582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1268/2544] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:27.529350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf60354-4ae8-4e38-929d-8b0f081f3c02 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:27.531231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-baf60354-4ae8-4e38-929d-8b0f081f3c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:27.548609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf60354-4ae8-4e38-929d-8b0f081f3c02 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:27.555761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:27.557258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:27.557779 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:27.558346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] PATH_INFO: `/v3/projects/e9e91ab36e7a40b7a70f6f69a8230622` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:27.560690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9e91ab36e7a40b7a70f6f69a8230622)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:27.564376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:27.650336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9e91ab36e7a40b7a70f6f69a8230622', 'request_id': 'req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:27.657683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9e91ab36e7a40b7a70f6f69a8230622', 'request_id': 'req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:27.668591 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9e91ab36e7a40b7a70f6f69a8230622', 'request_id': 'req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:27.683538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] The token cache is being invalidate because project e9e91ab36e7a40b7a70f6f69a8230622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:27.683837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:27.684516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:27.686165 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1277/2545] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:27 2022] DELETE /identity/v3/projects/e9e91ab36e7a40b7a70f6f69a8230622 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:27.690493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca06705-88bd-4c46-b861-1d75c8b198c3 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:27.701793 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca06705-88bd-4c46-b861-1d75c8b198c3 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:27.725126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:27.786166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145d415-a894-4d5c-a62d-59aac53f348a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:27.787239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145d415-a894-4d5c-a62d-59aac53f348a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:27.787696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145d415-a894-4d5c-a62d-59aac53f348a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:27.787972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145d415-a894-4d5c-a62d-59aac53f348a admin admin] PATH_INFO: `/v3/users/a21256eb490349398560f19100c58fde` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:27.789670 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145d415-a894-4d5c-a62d-59aac53f348a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a21256eb490349398560f19100c58fde)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:27.792555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145d415-a894-4d5c-a62d-59aac53f348a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:27.865603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3145d415-a894-4d5c-a62d-59aac53f348a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a21256eb490349398560f19100c58fde'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:27.866350 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3145d415-a894-4d5c-a62d-59aac53f348a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a21256eb490349398560f19100c58fde'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:27.867245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3145d415-a894-4d5c-a62d-59aac53f348a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a21256eb490349398560f19100c58fde'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:27.876734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3145d415-a894-4d5c-a62d-59aac53f348a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a21256eb490349398560f19100c58fde'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:27.891265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3145d415-a894-4d5c-a62d-59aac53f348a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a21256eb490349398560f19100c58fde'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:27.908067 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1269/2546] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:27 2022] DELETE /identity/v3/users/a21256eb490349398560f19100c58fde => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:28.330105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:28.333209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:28.393055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc722e-7f0c-4105-8c1c-7a2bd5dc5773 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:28.496670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ff78c-72ea-4ab7-b626-812881ebe1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:28.497573 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ff78c-72ea-4ab7-b626-812881ebe1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:28.498129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ff78c-72ea-4ab7-b626-812881ebe1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:28.498412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ff78c-72ea-4ab7-b626-812881ebe1b1 admin admin] PATH_INFO: `/v3/projects/675c68e578c249d2ab70e53b34cd2a08` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:28.502249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ff78c-72ea-4ab7-b626-812881ebe1b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=675c68e578c249d2ab70e53b34cd2a08)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:28.502249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ff78c-72ea-4ab7-b626-812881ebe1b1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:28.563868 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-861ff78c-72ea-4ab7-b626-812881ebe1b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '675c68e578c249d2ab70e53b34cd2a08', 'request_id': 'req-861ff78c-72ea-4ab7-b626-812881ebe1b1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:28.572720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-861ff78c-72ea-4ab7-b626-812881ebe1b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '675c68e578c249d2ab70e53b34cd2a08', 'request_id': 'req-861ff78c-72ea-4ab7-b626-812881ebe1b1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:28.585201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-861ff78c-72ea-4ab7-b626-812881ebe1b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '675c68e578c249d2ab70e53b34cd2a08', 'request_id': 'req-861ff78c-72ea-4ab7-b626-812881ebe1b1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:28.608555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-861ff78c-72ea-4ab7-b626-812881ebe1b1 admin admin] The token cache is being invalidate because project 675c68e578c249d2ab70e53b34cd2a08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:28.608822 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-861ff78c-72ea-4ab7-b626-812881ebe1b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:28.609606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-861ff78c-72ea-4ab7-b626-812881ebe1b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:28.611769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1278/2547] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:28 2022] DELETE /identity/v3/projects/675c68e578c249d2ab70e53b34cd2a08 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:28.618410 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f921286-df69-4af1-ac85-d814b03f4496 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:28.618410 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f921286-df69-4af1-ac85-d814b03f4496 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:28.618761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f921286-df69-4af1-ac85-d814b03f4496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:28.639607 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3f921286-df69-4af1-ac85-d814b03f4496 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:28.640909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3f921286-df69-4af1-ac85-d814b03f4496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:28.713913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1270/2548] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:28.723888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52796b3d-a9c4-4c53-9321-65ed4ed2fcac None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:28.724184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52796b3d-a9c4-4c53-9321-65ed4ed2fcac None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:28.724466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52796b3d-a9c4-4c53-9321-65ed4ed2fcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:28.738414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:28.746298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-52796b3d-a9c4-4c53-9321-65ed4ed2fcac None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:28.748388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-52796b3d-a9c4-4c53-9321-65ed4ed2fcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:28.760298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1279/2549] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:28.765467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f921286-df69-4af1-ac85-d814b03f4496 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:28.777111 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f921286-df69-4af1-ac85-d814b03f4496 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:28.782946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d74f9-47c2-419b-b60d-f3bc49cb47dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:28.783560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d74f9-47c2-419b-b60d-f3bc49cb47dd admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:28.783849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d74f9-47c2-419b-b60d-f3bc49cb47dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:28.784131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d74f9-47c2-419b-b60d-f3bc49cb47dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:28.784952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d74f9-47c2-419b-b60d-f3bc49cb47dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:28.785306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d74f9-47c2-419b-b60d-f3bc49cb47dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:28.787164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d74f9-47c2-419b-b60d-f3bc49cb47dd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:28.803335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1271/2550] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:11:28.808376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52796b3d-a9c4-4c53-9321-65ed4ed2fcac None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:28.820474 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52796b3d-a9c4-4c53-9321-65ed4ed2fcac None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:28.827358 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c64043-a672-4274-93a9-9f326f1a2cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:28.828049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c64043-a672-4274-93a9-9f326f1a2cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:28.828331 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c64043-a672-4274-93a9-9f326f1a2cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:28.828625 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c64043-a672-4274-93a9-9f326f1a2cd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:28.829367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c64043-a672-4274-93a9-9f326f1a2cd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:28.831383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c64043-a672-4274-93a9-9f326f1a2cd8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:28.869903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1280/2551] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:28 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:28.874369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d74f9-47c2-419b-b60d-f3bc49cb47dd admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:28.887115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d74f9-47c2-419b-b60d-f3bc49cb47dd admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:28.921751 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:28.959888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1891888-12a7-4861-8854-d2579bc42aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:28.960599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1891888-12a7-4861-8854-d2579bc42aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:28.960893 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1891888-12a7-4861-8854-d2579bc42aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:28.961362 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1891888-12a7-4861-8854-d2579bc42aa6 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:28.962117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1891888-12a7-4861-8854-d2579bc42aa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:28.964169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1891888-12a7-4861-8854-d2579bc42aa6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:28.986452 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1272/2552] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:28 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:28.991912 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c64043-a672-4274-93a9-9f326f1a2cd8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:29.012175 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c64043-a672-4274-93a9-9f326f1a2cd8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:29.021379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbca29-54d2-4a80-b176-67d63714e9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:29.022948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbca29-54d2-4a80-b176-67d63714e9fb admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.023493 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbca29-54d2-4a80-b176-67d63714e9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.024146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbca29-54d2-4a80-b176-67d63714e9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.025512 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbca29-54d2-4a80-b176-67d63714e9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:29.028747 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbca29-54d2-4a80-b176-67d63714e9fb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:29.049828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1281/2553] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:29.054134 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1891888-12a7-4861-8854-d2579bc42aa6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:29.065257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1891888-12a7-4861-8854-d2579bc42aa6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:29.071655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802acb21-3743-4b3b-b52f-a2c26e471ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:29.072405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802acb21-3743-4b3b-b52f-a2c26e471ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.072678 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802acb21-3743-4b3b-b52f-a2c26e471ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.072968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802acb21-3743-4b3b-b52f-a2c26e471ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.073772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802acb21-3743-4b3b-b52f-a2c26e471ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:29.075444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802acb21-3743-4b3b-b52f-a2c26e471ac6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:29.088927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1273/2554] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:29.093423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbca29-54d2-4a80-b176-67d63714e9fb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:29.104582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbca29-54d2-4a80-b176-67d63714e9fb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:29.109814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ce73a-b283-49e7-bce4-3679f263ef91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:29.110739 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ce73a-b283-49e7-bce4-3679f263ef91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.111021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ce73a-b283-49e7-bce4-3679f263ef91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.111320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ce73a-b283-49e7-bce4-3679f263ef91 admin admin] PATH_INFO: `/v3/projects/104b16becd304449a901719aaf6a30c9/users/b9ac2570177c44ad90e8800c795ee831/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.129544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ce73a-b283-49e7-bce4-3679f263ef91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=104b16becd304449a901719aaf6a30c9, user_id=b9ac2570177c44ad90e8800c795ee831, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:29.132322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ce73a-b283-49e7-bce4-3679f263ef91 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:29.152426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1282/2555] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:29 2022] PUT /identity/v3/projects/104b16becd304449a901719aaf6a30c9/users/b9ac2570177c44ad90e8800c795ee831/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:29.165537 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f2d50-28bc-4340-9332-c9354e60efd8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.165853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f2d50-28bc-4340-9332-c9354e60efd8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.166142 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f2d50-28bc-4340-9332-c9354e60efd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.208791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ab2f2d50-28bc-4340-9332-c9354e60efd8 None None] MFA Rules not processed for user `b9ac2570177c44ad90e8800c795ee831`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:29.210074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ab2f2d50-28bc-4340-9332-c9354e60efd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:29.306117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1274/2556] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:29.311875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ce73a-b283-49e7-bce4-3679f263ef91 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:29.366861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f2d50-28bc-4340-9332-c9354e60efd8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:29.368827 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ab2f2d50-28bc-4340-9332-c9354e60efd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:29.395344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ce73a-b283-49e7-bce4-3679f263ef91 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:29.400888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233acf25-1ce7-4622-9414-8ab1d74b4427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:29.401694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233acf25-1ce7-4622-9414-8ab1d74b4427 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.401986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233acf25-1ce7-4622-9414-8ab1d74b4427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.402335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233acf25-1ce7-4622-9414-8ab1d74b4427 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.403088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233acf25-1ce7-4622-9414-8ab1d74b4427 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:29.405119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233acf25-1ce7-4622-9414-8ab1d74b4427 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:29.447458 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1283/2557] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:29 2022] POST /identity/v3/projects => generated 389 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:29.452287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233acf25-1ce7-4622-9414-8ab1d74b4427 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:29.452671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f2d50-28bc-4340-9332-c9354e60efd8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:29.502345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:29.527954 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52436511-9145-4141-86d0-dc2ff6fc99f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:29.528981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52436511-9145-4141-86d0-dc2ff6fc99f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.529332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52436511-9145-4141-86d0-dc2ff6fc99f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.529757 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52436511-9145-4141-86d0-dc2ff6fc99f2 admin admin] PATH_INFO: `/v3/users/4055bbcb0f1f443197aef6c60f1afdb2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.531069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52436511-9145-4141-86d0-dc2ff6fc99f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4055bbcb0f1f443197aef6c60f1afdb2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:29.533334 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52436511-9145-4141-86d0-dc2ff6fc99f2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:29.537326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233acf25-1ce7-4622-9414-8ab1d74b4427 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:29.542606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0b1ab-5af8-4543-be80-c17bb60c81ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:29.543273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0b1ab-5af8-4543-be80-c17bb60c81ab admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.543794 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0b1ab-5af8-4543-be80-c17bb60c81ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.544364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0b1ab-5af8-4543-be80-c17bb60c81ab admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.544859 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0b1ab-5af8-4543-be80-c17bb60c81ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:29.547828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0b1ab-5af8-4543-be80-c17bb60c81ab admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:29.586542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1284/2558] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:29 2022] POST /identity/v3/users => generated 301 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:29.594619 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0b1ab-5af8-4543-be80-c17bb60c81ab admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:29.611627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0b1ab-5af8-4543-be80-c17bb60c81ab admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:29.616785 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d77d9d-dc3d-4c04-9253-0f8054947958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:29.617530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d77d9d-dc3d-4c04-9253-0f8054947958 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.617793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d77d9d-dc3d-4c04-9253-0f8054947958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.618105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d77d9d-dc3d-4c04-9253-0f8054947958 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.618977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d77d9d-dc3d-4c04-9253-0f8054947958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:29.620641 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d77d9d-dc3d-4c04-9253-0f8054947958 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:29.625962 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52436511-9145-4141-86d0-dc2ff6fc99f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4055bbcb0f1f443197aef6c60f1afdb2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:29.626797 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52436511-9145-4141-86d0-dc2ff6fc99f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4055bbcb0f1f443197aef6c60f1afdb2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:29.628602 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52436511-9145-4141-86d0-dc2ff6fc99f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4055bbcb0f1f443197aef6c60f1afdb2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:29.641930 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1285/2559] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:29.643870 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d77d9d-dc3d-4c04-9253-0f8054947958 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:29.644245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52436511-9145-4141-86d0-dc2ff6fc99f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4055bbcb0f1f443197aef6c60f1afdb2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:29.646629 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-29d77d9d-dc3d-4c04-9253-0f8054947958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:29.661880 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52436511-9145-4141-86d0-dc2ff6fc99f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4055bbcb0f1f443197aef6c60f1afdb2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:29.672855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d77d9d-dc3d-4c04-9253-0f8054947958 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:29.681997 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1538b-e063-43d2-a513-71fa477595c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:29.683432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1538b-e063-43d2-a513-71fa477595c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.683883 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1538b-e063-43d2-a513-71fa477595c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.684320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1538b-e063-43d2-a513-71fa477595c7 admin admin] PATH_INFO: `/v3/projects/40c57ea646f94e268863db070506f220/users/2f98c08720a74155a7ce475bd2a8f81e/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.684974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1275/2560] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:29 2022] DELETE /identity/v3/users/4055bbcb0f1f443197aef6c60f1afdb2 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:29.688778 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:29.689768 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d939cfa-bcda-4d47-97a5-1dc7034a1a5e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.690224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d939cfa-bcda-4d47-97a5-1dc7034a1a5e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.690592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d939cfa-bcda-4d47-97a5-1dc7034a1a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.705054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1538b-e063-43d2-a513-71fa477595c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c57ea646f94e268863db070506f220, user_id=2f98c08720a74155a7ce475bd2a8f81e, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:29.708666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1538b-e063-43d2-a513-71fa477595c7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:29.721960 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0d939cfa-bcda-4d47-97a5-1dc7034a1a5e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:29.723201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0d939cfa-bcda-4d47-97a5-1dc7034a1a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:29.728193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1286/2561] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:29 2022] PUT /identity/v3/projects/40c57ea646f94e268863db070506f220/users/2f98c08720a74155a7ce475bd2a8f81e/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:29.734053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1538b-e063-43d2-a513-71fa477595c7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:29.808784 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1276/2562] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:29.819409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d939cfa-bcda-4d47-97a5-1dc7034a1a5e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:29.821663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0d939cfa-bcda-4d47-97a5-1dc7034a1a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:29.843873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1538b-e063-43d2-a513-71fa477595c7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:29.850550 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575510b1-7a9c-49bf-8035-d74d3ff76af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:29.851511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575510b1-7a9c-49bf-8035-d74d3ff76af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.851838 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575510b1-7a9c-49bf-8035-d74d3ff76af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.852750 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575510b1-7a9c-49bf-8035-d74d3ff76af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.853525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575510b1-7a9c-49bf-8035-d74d3ff76af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:29.855236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575510b1-7a9c-49bf-8035-d74d3ff76af6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:29.879873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1287/2563] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:11:29.887303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575510b1-7a9c-49bf-8035-d74d3ff76af6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:29.905307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d939cfa-bcda-4d47-97a5-1dc7034a1a5e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:29.908770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575510b1-7a9c-49bf-8035-d74d3ff76af6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:29.910227 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb3008-8212-4fba-8394-4ca8c05160ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:29.913167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb3008-8212-4fba-8394-4ca8c05160ab service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.913614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb3008-8212-4fba-8394-4ca8c05160ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.913849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb3008-8212-4fba-8394-4ca8c05160ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.917852 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f3ade-2bbb-4cef-b4b4-b5ff14e1075e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:29.918901 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f3ade-2bbb-4cef-b4b4-b5ff14e1075e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.919190 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f3ade-2bbb-4cef-b4b4-b5ff14e1075e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.919589 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f3ade-2bbb-4cef-b4b4-b5ff14e1075e admin admin] PATH_INFO: `/v3/projects/40c57ea646f94e268863db070506f220/users/2f98c08720a74155a7ce475bd2a8f81e/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.923457 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f3ade-2bbb-4cef-b4b4-b5ff14e1075e admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c57ea646f94e268863db070506f220, user_id=2f98c08720a74155a7ce475bd2a8f81e, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:29.925429 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb3008-8212-4fba-8394-4ca8c05160ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:29.925699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f3ade-2bbb-4cef-b4b4-b5ff14e1075e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:29.926959 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb3008-8212-4fba-8394-4ca8c05160ab service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:29.937999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1277/2564] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:29.941725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1288/2565] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:29 2022] PUT /identity/v3/projects/40c57ea646f94e268863db070506f220/users/2f98c08720a74155a7ce475bd2a8f81e/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:29.950510 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c29c3-fed6-4f9c-8ad7-09952c0061f6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:29.950897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c29c3-fed6-4f9c-8ad7-09952c0061f6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:29.951250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c29c3-fed6-4f9c-8ad7-09952c0061f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:29.965272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:29.984171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-716c29c3-fed6-4f9c-8ad7-09952c0061f6 None None] MFA Rules not processed for user `2f98c08720a74155a7ce475bd2a8f81e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:29.985699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-716c29c3-fed6-4f9c-8ad7-09952c0061f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:30.060705 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1278/2566] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:29 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:30.072348 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f3ade-2bbb-4cef-b4b4-b5ff14e1075e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:30.074718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-990f3ade-2bbb-4cef-b4b4-b5ff14e1075e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:30.146188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f3ade-2bbb-4cef-b4b4-b5ff14e1075e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:30.150402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9427657-9dc4-45d8-b3a8-17a0bd9c20c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:30.151699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9427657-9dc4-45d8-b3a8-17a0bd9c20c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:30.151984 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9427657-9dc4-45d8-b3a8-17a0bd9c20c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:30.152258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9427657-9dc4-45d8-b3a8-17a0bd9c20c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:30.162052 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9427657-9dc4-45d8-b3a8-17a0bd9c20c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:30.163533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9427657-9dc4-45d8-b3a8-17a0bd9c20c6 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:30.174346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1289/2567] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:30.239508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c29c3-fed6-4f9c-8ad7-09952c0061f6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:30.241986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-716c29c3-fed6-4f9c-8ad7-09952c0061f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:30.319937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9427657-9dc4-45d8-b3a8-17a0bd9c20c6 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:30.334888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c29c3-fed6-4f9c-8ad7-09952c0061f6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:30.344383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:30.345283 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:30.345559 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:30.345862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] PATH_INFO: `/v3/projects/bf5d141b6c2249c4a93813a2ba75657a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:30.347130 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf5d141b6c2249c4a93813a2ba75657a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:30.349052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:30.393357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9427657-9dc4-45d8-b3a8-17a0bd9c20c6 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:30.406682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:30.411968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf5d141b6c2249c4a93813a2ba75657a', 'request_id': 'req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.419466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf5d141b6c2249c4a93813a2ba75657a', 'request_id': 'req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.428795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf5d141b6c2249c4a93813a2ba75657a', 'request_id': 'req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.444483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] The token cache is being invalidate because project bf5d141b6c2249c4a93813a2ba75657a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:30.444913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.445965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.447890 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1279/2568] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:30 2022] DELETE /identity/v3/projects/bf5d141b6c2249c4a93813a2ba75657a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:30.451780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:30.453302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:30.458605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b3700-f1ce-42a1-9586-44d37316e8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:30.459132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b3700-f1ce-42a1-9586-44d37316e8d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:30.459414 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b3700-f1ce-42a1-9586-44d37316e8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:30.459692 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b3700-f1ce-42a1-9586-44d37316e8d5 admin admin] PATH_INFO: `/v3/users/b9ac2570177c44ad90e8800c795ee831` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:30.460945 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b3700-f1ce-42a1-9586-44d37316e8d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9ac2570177c44ad90e8800c795ee831)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:30.463046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b3700-f1ce-42a1-9586-44d37316e8d5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:30.469192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4c879-a31a-4b3d-b2e8-399fd6ff7847 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:30.474730 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:30.475628 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:30.475910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:30.476195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] PATH_INFO: `/v3/users/57c01238c33144c082909aaf92939ed1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:30.477643 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57c01238c33144c082909aaf92939ed1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:30.479610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:30.497115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:30.519490 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0e1b3700-f1ce-42a1-9586-44d37316e8d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9ac2570177c44ad90e8800c795ee831'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.528578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0e1b3700-f1ce-42a1-9586-44d37316e8d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9ac2570177c44ad90e8800c795ee831'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.531480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57c01238c33144c082909aaf92939ed1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.532157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57c01238c33144c082909aaf92939ed1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.532806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57c01238c33144c082909aaf92939ed1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.536528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0e1b3700-f1ce-42a1-9586-44d37316e8d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9ac2570177c44ad90e8800c795ee831'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.537190 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0e1b3700-f1ce-42a1-9586-44d37316e8d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9ac2570177c44ad90e8800c795ee831'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.540258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57c01238c33144c082909aaf92939ed1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.550589 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0e1b3700-f1ce-42a1-9586-44d37316e8d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9ac2570177c44ad90e8800c795ee831'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.552830 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57c01238c33144c082909aaf92939ed1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:30.556999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1290/2569] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:30 2022] DELETE /identity/v3/users/b9ac2570177c44ad90e8800c795ee831 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:30.561873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eff424-3bf2-4fbb-bbb6-13a4a05d33d4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:30.562214 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eff424-3bf2-4fbb-bbb6-13a4a05d33d4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:30.562493 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eff424-3bf2-4fbb-bbb6-13a4a05d33d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:30.567131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1280/2570] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:30 2022] DELETE /identity/v3/users/57c01238c33144c082909aaf92939ed1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:30.579660 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-51eff424-3bf2-4fbb-bbb6-13a4a05d33d4 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:30.580874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-51eff424-3bf2-4fbb-bbb6-13a4a05d33d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:30.646791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1291/2571] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:30.657680 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:30.659909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:30.728379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e01cbc-bbbd-40bb-a30d-81b4cd6e09c1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:30.733396 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79399558-43d0-4ddf-a452-93887cb47947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:30.734459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79399558-43d0-4ddf-a452-93887cb47947 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:30.734813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79399558-43d0-4ddf-a452-93887cb47947 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:30.735158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79399558-43d0-4ddf-a452-93887cb47947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:30.747262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79399558-43d0-4ddf-a452-93887cb47947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:30.748838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79399558-43d0-4ddf-a452-93887cb47947 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:30.761684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1281/2572] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:31.257890 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eff424-3bf2-4fbb-bbb6-13a4a05d33d4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:31.259833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-51eff424-3bf2-4fbb-bbb6-13a4a05d33d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:31.282274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eff424-3bf2-4fbb-bbb6-13a4a05d33d4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:31.291049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:31.294505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:31.295699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:31.296333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2 admin admin] PATH_INFO: `/v3/projects/b23e33fed7be4995b73d3f94222e67fc` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:31.298948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b23e33fed7be4995b73d3f94222e67fc)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:31.307394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:31.308198 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79399558-43d0-4ddf-a452-93887cb47947 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:31.310199 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-79399558-43d0-4ddf-a452-93887cb47947 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:31.331124 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79399558-43d0-4ddf-a452-93887cb47947 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:31.337854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:31.338910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:31.339329 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:31.339611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] PATH_INFO: `/v3/projects/104b16becd304449a901719aaf6a30c9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:31.341279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=104b16becd304449a901719aaf6a30c9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:31.343724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:31.387710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:31.404380 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b23e33fed7be4995b73d3f94222e67fc', 'request_id': 'req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.415048 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b23e33fed7be4995b73d3f94222e67fc', 'request_id': 'req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.425736 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b23e33fed7be4995b73d3f94222e67fc', 'request_id': 'req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.432636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '104b16becd304449a901719aaf6a30c9', 'request_id': 'req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.440203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '104b16becd304449a901719aaf6a30c9', 'request_id': 'req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.443879 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2 admin admin] The token cache is being invalidate because project b23e33fed7be4995b73d3f94222e67fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:31.444177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.449571 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-03bb00fa-ec8e-4dd8-93ae-e8e926aa9ef2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.456824 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1292/2573] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:31 2022] DELETE /identity/v3/projects/b23e33fed7be4995b73d3f94222e67fc => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:31.457497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '104b16becd304449a901719aaf6a30c9', 'request_id': 'req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.458499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecaecf-1988-420c-a244-5ebeb13b39dc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:31.459449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecaecf-1988-420c-a244-5ebeb13b39dc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:31.459759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecaecf-1988-420c-a244-5ebeb13b39dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:31.482019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] The token cache is being invalidate because project 104b16becd304449a901719aaf6a30c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:31.482311 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.482999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.484584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1282/2574] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:31 2022] DELETE /identity/v3/projects/104b16becd304449a901719aaf6a30c9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:31.497490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:31.499005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:31.501010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3cecaecf-1988-420c-a244-5ebeb13b39dc None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:31.502543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3cecaecf-1988-420c-a244-5ebeb13b39dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:31.527066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:31.610827 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44d82d-5ce0-44fd-a5e4-96a490a9c5a1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:31.616072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:31.616363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1293/2575] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:31.617153 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:31.617522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:31.618562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] PATH_INFO: `/v3/users/2f98c08720a74155a7ce475bd2a8f81e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:31.619925 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f98c08720a74155a7ce475bd2a8f81e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:31.621960 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:31.624435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f4b53-bc03-41c5-8e43-2fbb2f488ad7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:31.624727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f4b53-bc03-41c5-8e43-2fbb2f488ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:31.625007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f4b53-bc03-41c5-8e43-2fbb2f488ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:31.644223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-107f4b53-bc03-41c5-8e43-2fbb2f488ad7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:31.645470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-107f4b53-bc03-41c5-8e43-2fbb2f488ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:31.659499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1294/2576] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:31.664786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f4b53-bc03-41c5-8e43-2fbb2f488ad7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:31.675860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f4b53-bc03-41c5-8e43-2fbb2f488ad7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:31.679683 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f98c08720a74155a7ce475bd2a8f81e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.681084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f98c08720a74155a7ce475bd2a8f81e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.681792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f98c08720a74155a7ce475bd2a8f81e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.682123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690da79c-0ff6-4738-a731-2a8c6b241caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:31.682811 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690da79c-0ff6-4738-a731-2a8c6b241caa admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:31.683285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690da79c-0ff6-4738-a731-2a8c6b241caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:31.683857 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690da79c-0ff6-4738-a731-2a8c6b241caa admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:31.684972 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690da79c-0ff6-4738-a731-2a8c6b241caa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:31.685296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690da79c-0ff6-4738-a731-2a8c6b241caa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:31.686913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690da79c-0ff6-4738-a731-2a8c6b241caa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:31.689484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f98c08720a74155a7ce475bd2a8f81e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.700135 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1295/2577] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:11:31.702959 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f98c08720a74155a7ce475bd2a8f81e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:31.704686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690da79c-0ff6-4738-a731-2a8c6b241caa admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:31.707158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-690da79c-0ff6-4738-a731-2a8c6b241caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:31.721587 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1283/2578] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:31 2022] DELETE /identity/v3/users/2f98c08720a74155a7ce475bd2a8f81e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:31.726452 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690da79c-0ff6-4738-a731-2a8c6b241caa admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:31.812491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e996906-1ad2-4770-85c3-190d479cdef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:31.813347 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e996906-1ad2-4770-85c3-190d479cdef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:31.813648 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e996906-1ad2-4770-85c3-190d479cdef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:31.813941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e996906-1ad2-4770-85c3-190d479cdef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:31.814650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e996906-1ad2-4770-85c3-190d479cdef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:31.816673 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e996906-1ad2-4770-85c3-190d479cdef0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:31.851682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1296/2579] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:31 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:31.856728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:31.874548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe136c-87e0-4799-8041-219d4434008d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:31.965087 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882af529-bd02-46bd-b13f-9b4367ac2de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:31.965725 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882af529-bd02-46bd-b13f-9b4367ac2de5 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:31.966058 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882af529-bd02-46bd-b13f-9b4367ac2de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:31.966292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882af529-bd02-46bd-b13f-9b4367ac2de5 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:31.967004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882af529-bd02-46bd-b13f-9b4367ac2de5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:31.970112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882af529-bd02-46bd-b13f-9b4367ac2de5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:31.997853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1284/2580] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:31 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:32.003663 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e996906-1ad2-4770-85c3-190d479cdef0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.015714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e996906-1ad2-4770-85c3-190d479cdef0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.021140 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd0d8f-c34b-4f36-846d-833a974c3020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.021711 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd0d8f-c34b-4f36-846d-833a974c3020 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.021875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd0d8f-c34b-4f36-846d-833a974c3020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.022156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd0d8f-c34b-4f36-846d-833a974c3020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.022912 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd0d8f-c34b-4f36-846d-833a974c3020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.025503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd0d8f-c34b-4f36-846d-833a974c3020 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.038121 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1297/2581] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:32.042220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882af529-bd02-46bd-b13f-9b4367ac2de5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.053040 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882af529-bd02-46bd-b13f-9b4367ac2de5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.062483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa12eb8-48b0-4e9e-9a0f-e9f87758f7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.063307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa12eb8-48b0-4e9e-9a0f-e9f87758f7c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.063653 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa12eb8-48b0-4e9e-9a0f-e9f87758f7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.064312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa12eb8-48b0-4e9e-9a0f-e9f87758f7c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.065033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa12eb8-48b0-4e9e-9a0f-e9f87758f7c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.067566 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa12eb8-48b0-4e9e-9a0f-e9f87758f7c5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.079344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1285/2582] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:32.083506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd0d8f-c34b-4f36-846d-833a974c3020 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.093809 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa12eb8-48b0-4e9e-9a0f-e9f87758f7c5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.097011 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8fa12eb8-48b0-4e9e-9a0f-e9f87758f7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:32.105479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd0d8f-c34b-4f36-846d-833a974c3020 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.112541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa12eb8-48b0-4e9e-9a0f-e9f87758f7c5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.123959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a561264-5aa0-4a11-b43b-13c11577c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.126608 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a561264-5aa0-4a11-b43b-13c11577c71e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.126828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a561264-5aa0-4a11-b43b-13c11577c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.127095 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a561264-5aa0-4a11-b43b-13c11577c71e admin admin] PATH_INFO: `/v3/projects/b4682464795f4eea94580e9b80a03dac/users/79b8180284044ae29b99417b501de82c/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.128208 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63f398-fe11-4dec-aa79-d4e02b76dd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.129178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63f398-fe11-4dec-aa79-d4e02b76dd99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.129528 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63f398-fe11-4dec-aa79-d4e02b76dd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.129909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63f398-fe11-4dec-aa79-d4e02b76dd99 admin admin] PATH_INFO: `/v3/projects/40c57ea646f94e268863db070506f220` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.131972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63f398-fe11-4dec-aa79-d4e02b76dd99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40c57ea646f94e268863db070506f220)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.142280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63f398-fe11-4dec-aa79-d4e02b76dd99 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.155663 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:32.161550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:32.162656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a561264-5aa0-4a11-b43b-13c11577c71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4682464795f4eea94580e9b80a03dac, user_id=79b8180284044ae29b99417b501de82c, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.166669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a561264-5aa0-4a11-b43b-13c11577c71e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.187272 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1298/2583] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:32 2022] PUT /identity/v3/projects/b4682464795f4eea94580e9b80a03dac/users/79b8180284044ae29b99417b501de82c/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:32.199690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa7211-426e-4f96-86f8-b8c0ce43948c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.200353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa7211-426e-4f96-86f8-b8c0ce43948c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.200353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa7211-426e-4f96-86f8-b8c0ce43948c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.215367 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4e63f398-fe11-4dec-aa79-d4e02b76dd99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40c57ea646f94e268863db070506f220', 'request_id': 'req-4e63f398-fe11-4dec-aa79-d4e02b76dd99'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:32.221313 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4e63f398-fe11-4dec-aa79-d4e02b76dd99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40c57ea646f94e268863db070506f220', 'request_id': 'req-4e63f398-fe11-4dec-aa79-d4e02b76dd99'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:32.228089 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b7aa7211-426e-4f96-86f8-b8c0ce43948c None None] MFA Rules not processed for user `79b8180284044ae29b99417b501de82c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:32.229326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b7aa7211-426e-4f96-86f8-b8c0ce43948c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:32.231775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4e63f398-fe11-4dec-aa79-d4e02b76dd99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40c57ea646f94e268863db070506f220', 'request_id': 'req-4e63f398-fe11-4dec-aa79-d4e02b76dd99'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:32.248723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4e63f398-fe11-4dec-aa79-d4e02b76dd99 admin admin] The token cache is being invalidate because project 40c57ea646f94e268863db070506f220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:32.248974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4e63f398-fe11-4dec-aa79-d4e02b76dd99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:32.249688 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4e63f398-fe11-4dec-aa79-d4e02b76dd99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:32.255115 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1286/2584] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:32 2022] DELETE /identity/v3/projects/40c57ea646f94e268863db070506f220 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:32.256634 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e727d-0c9c-4744-9ff0-f83c20972fde None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.256920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e727d-0c9c-4744-9ff0-f83c20972fde None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.257197 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e727d-0c9c-4744-9ff0-f83c20972fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.276164 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c78e727d-0c9c-4744-9ff0-f83c20972fde None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:32.277394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c78e727d-0c9c-4744-9ff0-f83c20972fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:32.304270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1299/2585] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:32.309111 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa7211-426e-4f96-86f8-b8c0ce43948c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.310646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b7aa7211-426e-4f96-86f8-b8c0ce43948c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:32.338876 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1287/2586] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:32.346731 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6911d63-796d-4e17-8c62-00df77c88047 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.347023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6911d63-796d-4e17-8c62-00df77c88047 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.347303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6911d63-796d-4e17-8c62-00df77c88047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.370969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b6911d63-796d-4e17-8c62-00df77c88047 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:32.372739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b6911d63-796d-4e17-8c62-00df77c88047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:32.384626 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1288/2587] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:32.386365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa7211-426e-4f96-86f8-b8c0ce43948c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.389316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6911d63-796d-4e17-8c62-00df77c88047 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.395287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299aeb8-e385-4b8e-bcda-b93c6018b3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.395287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299aeb8-e385-4b8e-bcda-b93c6018b3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.395627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299aeb8-e385-4b8e-bcda-b93c6018b3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.395719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299aeb8-e385-4b8e-bcda-b93c6018b3e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.396448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299aeb8-e385-4b8e-bcda-b93c6018b3e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.398487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299aeb8-e385-4b8e-bcda-b93c6018b3e5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.400142 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6911d63-796d-4e17-8c62-00df77c88047 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.405488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc92e2-e276-4fef-b671-182d1cd01616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.406211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc92e2-e276-4fef-b671-182d1cd01616 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.406496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc92e2-e276-4fef-b671-182d1cd01616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.406777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc92e2-e276-4fef-b671-182d1cd01616 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.407548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc92e2-e276-4fef-b671-182d1cd01616 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:32.407860 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc92e2-e276-4fef-b671-182d1cd01616 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.409488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc92e2-e276-4fef-b671-182d1cd01616 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.422454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1289/2588] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:11:32.428784 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc92e2-e276-4fef-b671-182d1cd01616 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.437967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1300/2589] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:32 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:32.441949 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299aeb8-e385-4b8e-bcda-b93c6018b3e5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.445406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc92e2-e276-4fef-b671-182d1cd01616 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.533287 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299aeb8-e385-4b8e-bcda-b93c6018b3e5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.537387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c015354-e111-4869-9a8f-a00f1c58f1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.538341 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c015354-e111-4869-9a8f-a00f1c58f1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.538660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c015354-e111-4869-9a8f-a00f1c58f1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.538935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c015354-e111-4869-9a8f-a00f1c58f1a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.539087 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa147f8c-9079-4749-aca3-2a2695f64f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.540168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa147f8c-9079-4749-aca3-2a2695f64f51 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.540374 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c015354-e111-4869-9a8f-a00f1c58f1a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.540537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa147f8c-9079-4749-aca3-2a2695f64f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.540537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa147f8c-9079-4749-aca3-2a2695f64f51 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.541073 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa147f8c-9079-4749-aca3-2a2695f64f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.541953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c015354-e111-4869-9a8f-a00f1c58f1a8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.543757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa147f8c-9079-4749-aca3-2a2695f64f51 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.554880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:32.578760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1301/2590] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:32 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:32.579345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1290/2591] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:32 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:32.579984 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa147f8c-9079-4749-aca3-2a2695f64f51 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.580696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c015354-e111-4869-9a8f-a00f1c58f1a8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.596182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa147f8c-9079-4749-aca3-2a2695f64f51 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.597172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:32.678799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c015354-e111-4869-9a8f-a00f1c58f1a8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.685125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb946e8b-b531-4e8d-9f5c-17d3d330e47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.686097 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb946e8b-b531-4e8d-9f5c-17d3d330e47e admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.686473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb946e8b-b531-4e8d-9f5c-17d3d330e47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.686828 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb946e8b-b531-4e8d-9f5c-17d3d330e47e admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.687664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb946e8b-b531-4e8d-9f5c-17d3d330e47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.689449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb946e8b-b531-4e8d-9f5c-17d3d330e47e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.694180 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c917993-9cce-4f76-a3c1-2a1582c089e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.694744 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c917993-9cce-4f76-a3c1-2a1582c089e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.695044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c917993-9cce-4f76-a3c1-2a1582c089e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.695357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c917993-9cce-4f76-a3c1-2a1582c089e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.697085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c917993-9cce-4f76-a3c1-2a1582c089e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.700848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c917993-9cce-4f76-a3c1-2a1582c089e4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.703716 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1291/2592] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:32.720540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb946e8b-b531-4e8d-9f5c-17d3d330e47e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.732928 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb946e8b-b531-4e8d-9f5c-17d3d330e47e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.738299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88d3b6-5f93-465b-879d-76a4f5ad7cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.739010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88d3b6-5f93-465b-879d-76a4f5ad7cab admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.739264 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88d3b6-5f93-465b-879d-76a4f5ad7cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.739587 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88d3b6-5f93-465b-879d-76a4f5ad7cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.740410 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88d3b6-5f93-465b-879d-76a4f5ad7cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.742935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88d3b6-5f93-465b-879d-76a4f5ad7cab admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.759316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1292/2593] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:32.760428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88d3b6-5f93-465b-879d-76a4f5ad7cab admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.763415 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1302/2594] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:32 2022] POST /identity/v3/users => generated 284 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:32.767505 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c917993-9cce-4f76-a3c1-2a1582c089e4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.783327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88d3b6-5f93-465b-879d-76a4f5ad7cab admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.785661 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c917993-9cce-4f76-a3c1-2a1582c089e4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.792243 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba9db5-2ab5-44ac-948f-908feafd82f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.793136 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d54e3-3afe-4ebc-b000-9b7503e81a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.793877 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba9db5-2ab5-44ac-948f-908feafd82f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.794384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba9db5-2ab5-44ac-948f-908feafd82f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.794742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba9db5-2ab5-44ac-948f-908feafd82f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.795279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d54e3-3afe-4ebc-b000-9b7503e81a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.795804 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba9db5-2ab5-44ac-948f-908feafd82f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.795932 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d54e3-3afe-4ebc-b000-9b7503e81a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.796590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d54e3-3afe-4ebc-b000-9b7503e81a23 admin admin] PATH_INFO: `/v3/projects/5816edfa344f4dc69745313b13dffcc9/users/d2fc1c1a275549dd88a67b1e423b9757/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.798580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba9db5-2ab5-44ac-948f-908feafd82f1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.816755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1303/2595] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:32.821893 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba9db5-2ab5-44ac-948f-908feafd82f1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.827363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d54e3-3afe-4ebc-b000-9b7503e81a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5816edfa344f4dc69745313b13dffcc9, user_id=d2fc1c1a275549dd88a67b1e423b9757, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.830229 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d54e3-3afe-4ebc-b000-9b7503e81a23 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.837616 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba9db5-2ab5-44ac-948f-908feafd82f1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.859256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1293/2596] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:32 2022] PUT /identity/v3/projects/5816edfa344f4dc69745313b13dffcc9/users/d2fc1c1a275549dd88a67b1e423b9757/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:32.862385 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ceaf0-6e2a-4dce-bffe-57e9c12cdc77 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.862941 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ceaf0-6e2a-4dce-bffe-57e9c12cdc77 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.863327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ceaf0-6e2a-4dce-bffe-57e9c12cdc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.907192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ef3ceaf0-6e2a-4dce-bffe-57e9c12cdc77 None None] MFA Rules not processed for user `d2fc1c1a275549dd88a67b1e423b9757`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:32.908513 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ef3ceaf0-6e2a-4dce-bffe-57e9c12cdc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:32.931504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94459f83-7647-4a61-a759-6b2d384db801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.932386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94459f83-7647-4a61-a759-6b2d384db801 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.932773 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94459f83-7647-4a61-a759-6b2d384db801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.932972 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94459f83-7647-4a61-a759-6b2d384db801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.934340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94459f83-7647-4a61-a759-6b2d384db801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:32.938071 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94459f83-7647-4a61-a759-6b2d384db801 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:32.952857 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1304/2597] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:32.956839 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94459f83-7647-4a61-a759-6b2d384db801 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:32.969219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94459f83-7647-4a61-a759-6b2d384db801 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:32.975030 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368c0a0-1b1c-4673-9e28-643dee84d3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:32.976050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368c0a0-1b1c-4673-9e28-643dee84d3be admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:32.977010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368c0a0-1b1c-4673-9e28-643dee84d3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:32.977010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368c0a0-1b1c-4673-9e28-643dee84d3be admin admin] PATH_INFO: `/v3/projects/7134b15c7b8c4764b7fb480b6d081cd1/users/e3a53f9b85c64774a55e6b4046c58afe/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:32.998055 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368c0a0-1b1c-4673-9e28-643dee84d3be admin admin] RBAC: Authorizing `identity:create_grant(project_id=7134b15c7b8c4764b7fb480b6d081cd1, user_id=e3a53f9b85c64774a55e6b4046c58afe, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.001379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368c0a0-1b1c-4673-9e28-643dee84d3be admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.003741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1294/2598] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:33.008366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ceaf0-6e2a-4dce-bffe-57e9c12cdc77 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:33.031204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1305/2599] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:32 2022] PUT /identity/v3/projects/7134b15c7b8c4764b7fb480b6d081cd1/users/e3a53f9b85c64774a55e6b4046c58afe/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:33.032848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d3fd0-43ee-43bd-b0c0-c9163a77bce7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.033136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d3fd0-43ee-43bd-b0c0-c9163a77bce7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.033412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d3fd0-43ee-43bd-b0c0-c9163a77bce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.033591 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ceaf0-6e2a-4dce-bffe-57e9c12cdc77 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:33.070959 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ea9d3fd0-43ee-43bd-b0c0-c9163a77bce7 None None] MFA Rules not processed for user `e3a53f9b85c64774a55e6b4046c58afe`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:33.072282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ea9d3fd0-43ee-43bd-b0c0-c9163a77bce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:33.168831 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:33.179477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:33.195820 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30bbda-fe35-4a6c-8ec8-26b3eb91ab08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:33.197281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30bbda-fe35-4a6c-8ec8-26b3eb91ab08 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.198010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30bbda-fe35-4a6c-8ec8-26b3eb91ab08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.198394 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30bbda-fe35-4a6c-8ec8-26b3eb91ab08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.200200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30bbda-fe35-4a6c-8ec8-26b3eb91ab08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.203733 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30bbda-fe35-4a6c-8ec8-26b3eb91ab08 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.250706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1306/2600] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:33.257677 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d3fd0-43ee-43bd-b0c0-c9163a77bce7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:33.265981 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1295/2601] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:33 2022] POST /identity/v3/projects => generated 375 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:33.271990 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30bbda-fe35-4a6c-8ec8-26b3eb91ab08 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:33.272242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d3fd0-43ee-43bd-b0c0-c9163a77bce7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:33.354217 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a39e4b-8495-4d3f-8a93-c4739d1bd09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:33.355454 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a39e4b-8495-4d3f-8a93-c4739d1bd09c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.355910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a39e4b-8495-4d3f-8a93-c4739d1bd09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.356196 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a39e4b-8495-4d3f-8a93-c4739d1bd09c admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.357101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a39e4b-8495-4d3f-8a93-c4739d1bd09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.359159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a39e4b-8495-4d3f-8a93-c4739d1bd09c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.369538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30bbda-fe35-4a6c-8ec8-26b3eb91ab08 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:33.376486 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea1b0d-731f-4bce-9c5b-18face00140d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:33.377114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea1b0d-731f-4bce-9c5b-18face00140d admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.377510 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea1b0d-731f-4bce-9c5b-18face00140d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.378160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea1b0d-731f-4bce-9c5b-18face00140d admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.378882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea1b0d-731f-4bce-9c5b-18face00140d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.381244 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea1b0d-731f-4bce-9c5b-18face00140d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.406580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1307/2602] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:33 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:33.410275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a39e4b-8495-4d3f-8a93-c4739d1bd09c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:33.410847 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1296/2603] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:33 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:33.414935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea1b0d-731f-4bce-9c5b-18face00140d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:33.426990 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea1b0d-731f-4bce-9c5b-18face00140d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:33.494523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9e309-5336-4929-88a1-744322ee3d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:33.495216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9e309-5336-4929-88a1-744322ee3d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.495659 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9e309-5336-4929-88a1-744322ee3d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.495958 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9e309-5336-4929-88a1-744322ee3d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.496695 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9e309-5336-4929-88a1-744322ee3d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.498390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9e309-5336-4929-88a1-744322ee3d54 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.500506 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a39e4b-8495-4d3f-8a93-c4739d1bd09c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:33.510665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5286f-ed61-4a62-bc24-5147602b7557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:33.511417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5286f-ed61-4a62-bc24-5147602b7557 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.511723 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5286f-ed61-4a62-bc24-5147602b7557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.512014 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5286f-ed61-4a62-bc24-5147602b7557 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.512818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1297/2604] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:11:33.513032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5286f-ed61-4a62-bc24-5147602b7557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.515093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5286f-ed61-4a62-bc24-5147602b7557 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.518432 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9e309-5336-4929-88a1-744322ee3d54 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:33.534490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9e309-5336-4929-88a1-744322ee3d54 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:33.544266 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1308/2605] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:33 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:33.544552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a800e0-d219-4332-843c-8235aa09e00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:33.544552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a800e0-d219-4332-843c-8235aa09e00a admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.544552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a800e0-d219-4332-843c-8235aa09e00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.544552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a800e0-d219-4332-843c-8235aa09e00a admin admin] PATH_INFO: `/v3/projects/830e34cc42f740f79612cb1115b26bda/users/d21b4ccb8f0943efa052180ad44e6cc0/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.545668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5286f-ed61-4a62-bc24-5147602b7557 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:33.559562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5286f-ed61-4a62-bc24-5147602b7557 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:33.561418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a800e0-d219-4332-843c-8235aa09e00a admin admin] RBAC: Authorizing `identity:create_grant(project_id=830e34cc42f740f79612cb1115b26bda, user_id=d21b4ccb8f0943efa052180ad44e6cc0, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.564403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a800e0-d219-4332-843c-8235aa09e00a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.565159 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c4108-61d1-4a7b-b1d7-7883f53193ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:33.566227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c4108-61d1-4a7b-b1d7-7883f53193ba admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.566678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c4108-61d1-4a7b-b1d7-7883f53193ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.567062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c4108-61d1-4a7b-b1d7-7883f53193ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.568002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c4108-61d1-4a7b-b1d7-7883f53193ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.571088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c4108-61d1-4a7b-b1d7-7883f53193ba admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.586419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1298/2606] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:33 2022] PUT /identity/v3/projects/830e34cc42f740f79612cb1115b26bda/users/d21b4ccb8f0943efa052180ad44e6cc0/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:33.587009 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1309/2607] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:33.587618 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a800e0-d219-4332-843c-8235aa09e00a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:33.588970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c4108-61d1-4a7b-b1d7-7883f53193ba admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:33.601537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c4108-61d1-4a7b-b1d7-7883f53193ba admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:33.649273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:33.657522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:33.688607 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098821d-8d2d-4189-9ac4-6deab2d2c65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:33.689529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098821d-8d2d-4189-9ac4-6deab2d2c65f admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.689846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098821d-8d2d-4189-9ac4-6deab2d2c65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.690122 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098821d-8d2d-4189-9ac4-6deab2d2c65f admin admin] PATH_INFO: `/v3/projects/14f57cdc9c034d31b8a2994b5cb196c0/users/1844a0395afa4011a7698a0358db30eb/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.710114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a800e0-d219-4332-843c-8235aa09e00a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:33.711447 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098821d-8d2d-4189-9ac4-6deab2d2c65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f57cdc9c034d31b8a2994b5cb196c0, user_id=1844a0395afa4011a7698a0358db30eb, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.714938 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098821d-8d2d-4189-9ac4-6deab2d2c65f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.718596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a77ec-2329-4f81-8515-64d4e45d2c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:33.719849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a77ec-2329-4f81-8515-64d4e45d2c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.720288 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a77ec-2329-4f81-8515-64d4e45d2c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.720953 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a77ec-2329-4f81-8515-64d4e45d2c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.722839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a77ec-2329-4f81-8515-64d4e45d2c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.727227 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a77ec-2329-4f81-8515-64d4e45d2c2b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.735373 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1310/2608] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:33 2022] PUT /identity/v3/projects/14f57cdc9c034d31b8a2994b5cb196c0/users/1844a0395afa4011a7698a0358db30eb/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:33.739900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098821d-8d2d-4189-9ac4-6deab2d2c65f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:33.744888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1299/2609] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:33.750886 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a77ec-2329-4f81-8515-64d4e45d2c2b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:33.763315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a77ec-2329-4f81-8515-64d4e45d2c2b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:33.823137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098821d-8d2d-4189-9ac4-6deab2d2c65f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:33.827483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d64d7-a174-4970-9d82-88c236c0a59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:33.827907 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb964318-1df8-4532-b7bf-df04cb2b9278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:33.828571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d64d7-a174-4970-9d82-88c236c0a59f admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.828705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb964318-1df8-4532-b7bf-df04cb2b9278 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.828833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb964318-1df8-4532-b7bf-df04cb2b9278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.828929 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d64d7-a174-4970-9d82-88c236c0a59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.829108 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb964318-1df8-4532-b7bf-df04cb2b9278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.829242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d64d7-a174-4970-9d82-88c236c0a59f admin admin] PATH_INFO: `/v3/projects/830e34cc42f740f79612cb1115b26bda/users/d21b4ccb8f0943efa052180ad44e6cc0/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.829866 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb964318-1df8-4532-b7bf-df04cb2b9278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.831499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d64d7-a174-4970-9d82-88c236c0a59f admin admin] RBAC: Authorizing `identity:create_grant(project_id=830e34cc42f740f79612cb1115b26bda, user_id=d21b4ccb8f0943efa052180ad44e6cc0, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.831971 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb964318-1df8-4532-b7bf-df04cb2b9278 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.834276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d64d7-a174-4970-9d82-88c236c0a59f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.847574 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1300/2610] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:33 2022] PUT /identity/v3/projects/830e34cc42f740f79612cb1115b26bda/users/d21b4ccb8f0943efa052180ad44e6cc0/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:33.851443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1311/2611] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:33.855256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d64d7-a174-4970-9d82-88c236c0a59f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:33.861446 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27914e79-fa6d-47bf-937b-c0b7659b5e05 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.862022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27914e79-fa6d-47bf-937b-c0b7659b5e05 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.862517 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27914e79-fa6d-47bf-937b-c0b7659b5e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.902860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-27914e79-fa6d-47bf-937b-c0b7659b5e05 None None] MFA Rules not processed for user `d21b4ccb8f0943efa052180ad44e6cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:33.905065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-27914e79-fa6d-47bf-937b-c0b7659b5e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:33.959721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d64d7-a174-4970-9d82-88c236c0a59f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:33.970673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c52a7-359a-4613-816b-c99aa7fb8efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:33.972162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c52a7-359a-4613-816b-c99aa7fb8efa admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:33.972611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c52a7-359a-4613-816b-c99aa7fb8efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:33.973073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c52a7-359a-4613-816b-c99aa7fb8efa admin admin] PATH_INFO: `/v3/projects/14f57cdc9c034d31b8a2994b5cb196c0/users/1844a0395afa4011a7698a0358db30eb/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:33.976740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c52a7-359a-4613-816b-c99aa7fb8efa admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f57cdc9c034d31b8a2994b5cb196c0, user_id=1844a0395afa4011a7698a0358db30eb, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:33.980988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c52a7-359a-4613-816b-c99aa7fb8efa admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:33.994432 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1301/2612] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:33 2022] PUT /identity/v3/projects/14f57cdc9c034d31b8a2994b5cb196c0/users/1844a0395afa4011a7698a0358db30eb/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:34.038151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1312/2613] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:34.051642 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c52a7-359a-4613-816b-c99aa7fb8efa admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:34.054765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ec8c52a7-359a-4613-816b-c99aa7fb8efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:34.075590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d51ae9-9a65-4c69-86f7-c400eedda5e9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:34.075977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d51ae9-9a65-4c69-86f7-c400eedda5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:34.076483 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d51ae9-9a65-4c69-86f7-c400eedda5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:34.127072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-40d51ae9-9a65-4c69-86f7-c400eedda5e9 None None] MFA Rules not processed for user `1844a0395afa4011a7698a0358db30eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:34.129992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-40d51ae9-9a65-4c69-86f7-c400eedda5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:34.185551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c52a7-359a-4613-816b-c99aa7fb8efa admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:34.190480 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:34.196335 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a09903-a4f1-45a9-af40-91f9d435260c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:34.197074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a09903-a4f1-45a9-af40-91f9d435260c service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:34.197360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a09903-a4f1-45a9-af40-91f9d435260c service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:34.197656 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a09903-a4f1-45a9-af40-91f9d435260c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:34.233997 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:34.264873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1313/2614] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:34 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:34.314035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a09903-a4f1-45a9-af40-91f9d435260c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:34.315569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a09903-a4f1-45a9-af40-91f9d435260c service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:34.327951 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1302/2615] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:34.352012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d51ae9-9a65-4c69-86f7-c400eedda5e9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:34.353799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-40d51ae9-9a65-4c69-86f7-c400eedda5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:34.367653 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d51ae9-9a65-4c69-86f7-c400eedda5e9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:34.379607 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bc4e3-5078-4eaa-9e72-14d00102fc2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:34.380367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bc4e3-5078-4eaa-9e72-14d00102fc2a service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:34.380646 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bc4e3-5078-4eaa-9e72-14d00102fc2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:34.380915 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bc4e3-5078-4eaa-9e72-14d00102fc2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:34.392576 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9bc4e3-5078-4eaa-9e72-14d00102fc2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:34.394123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9bc4e3-5078-4eaa-9e72-14d00102fc2a service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:34.409884 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1314/2616] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:34 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:34.472209 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a09903-a4f1-45a9-af40-91f9d435260c service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:34.499983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a09903-a4f1-45a9-af40-91f9d435260c service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:34.511392 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e310a5-88c7-486c-bb2d-2b262c4d9002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:34.512528 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e310a5-88c7-486c-bb2d-2b262c4d9002 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:34.513021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e310a5-88c7-486c-bb2d-2b262c4d9002 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:34.513500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e310a5-88c7-486c-bb2d-2b262c4d9002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:34.688301 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e310a5-88c7-486c-bb2d-2b262c4d9002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:34.688533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bc4e3-5078-4eaa-9e72-14d00102fc2a service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:34.691216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e310a5-88c7-486c-bb2d-2b262c4d9002 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:34.703572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1303/2617] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:34.718951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bc4e3-5078-4eaa-9e72-14d00102fc2a service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:34.733427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d80146-5db9-49ff-8f5e-e3242846d955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:34.738918 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d80146-5db9-49ff-8f5e-e3242846d955 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:34.739324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d80146-5db9-49ff-8f5e-e3242846d955 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:34.739598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d80146-5db9-49ff-8f5e-e3242846d955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:34.848854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d80146-5db9-49ff-8f5e-e3242846d955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:34.850700 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d80146-5db9-49ff-8f5e-e3242846d955 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:34.869324 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1315/2618] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:37.929538 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e310a5-88c7-486c-bb2d-2b262c4d9002 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:38.034128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e310a5-88c7-486c-bb2d-2b262c4d9002 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:38.039873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:38.040763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:38.041512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:38.041512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] PATH_INFO: `/v3/users/79b8180284044ae29b99417b501de82c` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:38.042914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b8180284044ae29b99417b501de82c)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:38.045214 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:38.115439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b8180284044ae29b99417b501de82c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:38.116110 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b8180284044ae29b99417b501de82c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:38.116902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b8180284044ae29b99417b501de82c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:38.130024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b8180284044ae29b99417b501de82c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:38.134255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:38.156348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b8180284044ae29b99417b501de82c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:38.171614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1304/2619] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:37 2022] DELETE /identity/v3/users/79b8180284044ae29b99417b501de82c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:38.176693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735d04e-cea5-4913-808d-8ed1b2032a69 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:38.177783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735d04e-cea5-4913-808d-8ed1b2032a69 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:38.177783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735d04e-cea5-4913-808d-8ed1b2032a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:38.207376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5735d04e-cea5-4913-808d-8ed1b2032a69 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:38.208640 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5735d04e-cea5-4913-808d-8ed1b2032a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:38.292556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1316/2620] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:38.303601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:38.305179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:38.373325 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b255bb5-874c-4bd5-8274-96d2cb9ef308 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:38.377269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1bb4d-7765-45e4-b138-1036d36af653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:38.378327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1bb4d-7765-45e4-b138-1036d36af653 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:38.378608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1bb4d-7765-45e4-b138-1036d36af653 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:38.378882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1bb4d-7765-45e4-b138-1036d36af653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:38.402885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1bb4d-7765-45e4-b138-1036d36af653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:38.402885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1bb4d-7765-45e4-b138-1036d36af653 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:38.412274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1305/2621] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:38.715843 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735d04e-cea5-4913-808d-8ed1b2032a69 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:38.718439 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5735d04e-cea5-4913-808d-8ed1b2032a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:38.731058 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735d04e-cea5-4913-808d-8ed1b2032a69 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:38.741386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:38.743556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:38.743917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:38.744213 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] PATH_INFO: `/v3/projects/b4682464795f4eea94580e9b80a03dac` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:38.745472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4682464795f4eea94580e9b80a03dac)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:38.747479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:38.804112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:38.805299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4682464795f4eea94580e9b80a03dac', 'request_id': 'req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:38.812313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4682464795f4eea94580e9b80a03dac', 'request_id': 'req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:38.821032 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4682464795f4eea94580e9b80a03dac', 'request_id': 'req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:38.834377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] The token cache is being invalidate because project b4682464795f4eea94580e9b80a03dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:38.835188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:38.835381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:38.837011 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1317/2622] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:38 2022] DELETE /identity/v3/projects/b4682464795f4eea94580e9b80a03dac => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:38.841418 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1bb4d-7765-45e4-b138-1036d36af653 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:38.853616 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1bb4d-7765-45e4-b138-1036d36af653 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:38.945466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:38.946296 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:38.946576 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:38.946851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] PATH_INFO: `/v3/users/d2fc1c1a275549dd88a67b1e423b9757` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:38.948217 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2fc1c1a275549dd88a67b1e423b9757)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:38.950199 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:39.011926 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2fc1c1a275549dd88a67b1e423b9757'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.012679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2fc1c1a275549dd88a67b1e423b9757'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.013766 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2fc1c1a275549dd88a67b1e423b9757'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.023988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2fc1c1a275549dd88a67b1e423b9757'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.048234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2fc1c1a275549dd88a67b1e423b9757'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.060872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:39.070255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1306/2623] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:38 2022] DELETE /identity/v3/users/d2fc1c1a275549dd88a67b1e423b9757 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:39.434845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:39.436966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:39.452882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051e16e-0bb6-4cb9-a4e0-a138f0ddd511 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:39.550776 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:39.552148 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:39.552593 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:39.552873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] PATH_INFO: `/v3/projects/5816edfa344f4dc69745313b13dffcc9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:39.554732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5816edfa344f4dc69745313b13dffcc9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:39.557468 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:39.630894 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5816edfa344f4dc69745313b13dffcc9', 'request_id': 'req-3295f383-3538-440a-a917-f610c3316274'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.641985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5816edfa344f4dc69745313b13dffcc9', 'request_id': 'req-3295f383-3538-440a-a917-f610c3316274'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.654322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5816edfa344f4dc69745313b13dffcc9', 'request_id': 'req-3295f383-3538-440a-a917-f610c3316274'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.673960 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] The token cache is being invalidate because project 5816edfa344f4dc69745313b13dffcc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:39.674320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.675062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.677209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1318/2624] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:39 2022] DELETE /identity/v3/projects/5816edfa344f4dc69745313b13dffcc9 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:39.681272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:39.683113 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:39.698442 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7087d-f017-49e2-8f77-6cb99851e8e0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:39.799126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:39.800109 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:39.800389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:39.800696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] PATH_INFO: `/v3/users/d21b4ccb8f0943efa052180ad44e6cc0` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:39.802177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d21b4ccb8f0943efa052180ad44e6cc0)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:39.804254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:39.866300 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd21b4ccb8f0943efa052180ad44e6cc0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.866982 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd21b4ccb8f0943efa052180ad44e6cc0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.867822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd21b4ccb8f0943efa052180ad44e6cc0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.876389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd21b4ccb8f0943efa052180ad44e6cc0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.891467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd21b4ccb8f0943efa052180ad44e6cc0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:39.907240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1307/2625] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:39 2022] DELETE /identity/v3/users/d21b4ccb8f0943efa052180ad44e6cc0 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:40.174461 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:40.176045 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:40.191090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295f383-3538-440a-a917-f610c3316274 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:40.259486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bd722-fc8a-410b-81fe-3d49d37435c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:40.260044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bd722-fc8a-410b-81fe-3d49d37435c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:40.260317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bd722-fc8a-410b-81fe-3d49d37435c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:40.260613 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bd722-fc8a-410b-81fe-3d49d37435c1 admin admin] PATH_INFO: `/v3/users/e3a53f9b85c64774a55e6b4046c58afe` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:40.261895 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5bd722-fc8a-410b-81fe-3d49d37435c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3a53f9b85c64774a55e6b4046c58afe)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:40.264103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5bd722-fc8a-410b-81fe-3d49d37435c1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:40.271238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:40.326390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:40.327196 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:40.345360 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c5bd722-fc8a-410b-81fe-3d49d37435c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3a53f9b85c64774a55e6b4046c58afe'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:40.345862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b8ad9-63a0-41b3-827b-1fe8c83fe948 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:40.351968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91d6e0-c237-48d8-92ce-059a3ae0594b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:40.354309 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91d6e0-c237-48d8-92ce-059a3ae0594b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:40.355219 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91d6e0-c237-48d8-92ce-059a3ae0594b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:40.355996 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91d6e0-c237-48d8-92ce-059a3ae0594b admin admin] PATH_INFO: `/v3/projects/830e34cc42f740f79612cb1115b26bda` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:40.358137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c5bd722-fc8a-410b-81fe-3d49d37435c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3a53f9b85c64774a55e6b4046c58afe'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:40.359244 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91d6e0-c237-48d8-92ce-059a3ae0594b admin admin] RBAC: Authorizing `identity:delete_project(project_id=830e34cc42f740f79612cb1115b26bda)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:40.362310 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91d6e0-c237-48d8-92ce-059a3ae0594b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:40.378769 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c5bd722-fc8a-410b-81fe-3d49d37435c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3a53f9b85c64774a55e6b4046c58afe'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:40.380028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c5bd722-fc8a-410b-81fe-3d49d37435c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3a53f9b85c64774a55e6b4046c58afe'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:40.407710 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c5bd722-fc8a-410b-81fe-3d49d37435c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3a53f9b85c64774a55e6b4046c58afe'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:40.415770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1319/2626] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:40 2022] DELETE /identity/v3/users/e3a53f9b85c64774a55e6b4046c58afe => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:40.420763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbbd28-8737-4f79-95eb-1a850d426a98 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:40.421063 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbbd28-8737-4f79-95eb-1a850d426a98 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:40.430987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbbd28-8737-4f79-95eb-1a850d426a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:40.431148 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d91d6e0-c237-48d8-92ce-059a3ae0594b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830e34cc42f740f79612cb1115b26bda', 'request_id': 'req-2d91d6e0-c237-48d8-92ce-059a3ae0594b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:40.435028 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d91d6e0-c237-48d8-92ce-059a3ae0594b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830e34cc42f740f79612cb1115b26bda', 'request_id': 'req-2d91d6e0-c237-48d8-92ce-059a3ae0594b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:40.439758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-42bbbd28-8737-4f79-95eb-1a850d426a98 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:40.441090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-42bbbd28-8737-4f79-95eb-1a850d426a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:40.442346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d91d6e0-c237-48d8-92ce-059a3ae0594b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830e34cc42f740f79612cb1115b26bda', 'request_id': 'req-2d91d6e0-c237-48d8-92ce-059a3ae0594b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:40.456178 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:40.463744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d91d6e0-c237-48d8-92ce-059a3ae0594b admin admin] The token cache is being invalidate because project 830e34cc42f740f79612cb1115b26bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:40.464727 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d91d6e0-c237-48d8-92ce-059a3ae0594b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:40.465800 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d91d6e0-c237-48d8-92ce-059a3ae0594b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:40.473758 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1308/2627] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:40 2022] DELETE /identity/v3/projects/830e34cc42f740f79612cb1115b26bda => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:40.476791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174bc57-6988-49b7-b76e-40199c8ccca8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:40.477080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174bc57-6988-49b7-b76e-40199c8ccca8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:40.477358 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174bc57-6988-49b7-b76e-40199c8ccca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:40.499551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1174bc57-6988-49b7-b76e-40199c8ccca8 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:40.500670 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1174bc57-6988-49b7-b76e-40199c8ccca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:40.515900 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1320/2628] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:40.526626 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbbd28-8737-4f79-95eb-1a850d426a98 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:40.528146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-42bbbd28-8737-4f79-95eb-1a850d426a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:40.593812 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1309/2629] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:40.596807 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbbd28-8737-4f79-95eb-1a850d426a98 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:40.601231 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b9885-f995-4705-97de-b79596417c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:40.601951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b9885-f995-4705-97de-b79596417c56 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:40.602299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b9885-f995-4705-97de-b79596417c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:40.602719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b9885-f995-4705-97de-b79596417c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:40.608585 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7d629-2be1-4ac4-98d3-95d2693dd83a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:40.609073 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7d629-2be1-4ac4-98d3-95d2693dd83a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:40.609556 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7d629-2be1-4ac4-98d3-95d2693dd83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:40.618067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b9885-f995-4705-97de-b79596417c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:40.619728 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b9885-f995-4705-97de-b79596417c56 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:40.632034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1321/2630] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:40.639281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-64b7d629-2be1-4ac4-98d3-95d2693dd83a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:40.641924 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-64b7d629-2be1-4ac4-98d3-95d2693dd83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:40.656560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1310/2631] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:40.661917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b9885-f995-4705-97de-b79596417c56 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:40.673475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b9885-f995-4705-97de-b79596417c56 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:40.681867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a031d9-34d8-4a43-a897-5cf2d4eb4fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:40.683259 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a031d9-34d8-4a43-a897-5cf2d4eb4fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:40.683926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a031d9-34d8-4a43-a897-5cf2d4eb4fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:40.684448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a031d9-34d8-4a43-a897-5cf2d4eb4fa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:40.686007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a031d9-34d8-4a43-a897-5cf2d4eb4fa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:40.686814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a031d9-34d8-4a43-a897-5cf2d4eb4fa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:40.690078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a031d9-34d8-4a43-a897-5cf2d4eb4fa9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:40.717106 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1322/2632] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:11:40.723571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7d629-2be1-4ac4-98d3-95d2693dd83a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:40.737859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7d629-2be1-4ac4-98d3-95d2693dd83a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:40.751400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f54c4-6679-411d-afb4-a2348a51fc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:40.752322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f54c4-6679-411d-afb4-a2348a51fc82 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:40.753306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f54c4-6679-411d-afb4-a2348a51fc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:40.753707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f54c4-6679-411d-afb4-a2348a51fc82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:40.754693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f54c4-6679-411d-afb4-a2348a51fc82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:40.757426 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f54c4-6679-411d-afb4-a2348a51fc82 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:40.804899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1311/2633] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:40 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:40.806026 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a031d9-34d8-4a43-a897-5cf2d4eb4fa9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:40.826948 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a031d9-34d8-4a43-a897-5cf2d4eb4fa9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:40.850885 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:40.941758 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb097d6-eede-433e-8bca-afc26a52792e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:40.942521 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb097d6-eede-433e-8bca-afc26a52792e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:40.943316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb097d6-eede-433e-8bca-afc26a52792e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:40.943632 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb097d6-eede-433e-8bca-afc26a52792e admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:40.944580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb097d6-eede-433e-8bca-afc26a52792e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:40.947386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb097d6-eede-433e-8bca-afc26a52792e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:40.972112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1323/2634] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:40 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:40.976363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f54c4-6679-411d-afb4-a2348a51fc82 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:40.987687 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f54c4-6679-411d-afb4-a2348a51fc82 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:40.999299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03eaab2-be40-431d-b6a6-691f5265c207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:41.000126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03eaab2-be40-431d-b6a6-691f5265c207 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:41.000413 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03eaab2-be40-431d-b6a6-691f5265c207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:41.000747 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03eaab2-be40-431d-b6a6-691f5265c207 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:41.002010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03eaab2-be40-431d-b6a6-691f5265c207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:41.004031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03eaab2-be40-431d-b6a6-691f5265c207 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:41.018286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1312/2635] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:41.022338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb097d6-eede-433e-8bca-afc26a52792e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:41.028285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03eaab2-be40-431d-b6a6-691f5265c207 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:41.031439 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d03eaab2-be40-431d-b6a6-691f5265c207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:41.043850 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb097d6-eede-433e-8bca-afc26a52792e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:41.051308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90012a-30a9-4440-b495-58a30fb215b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:41.052178 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90012a-30a9-4440-b495-58a30fb215b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:41.052526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90012a-30a9-4440-b495-58a30fb215b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:41.052870 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90012a-30a9-4440-b495-58a30fb215b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:41.053954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90012a-30a9-4440-b495-58a30fb215b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:41.054382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03eaab2-be40-431d-b6a6-691f5265c207 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:41.055775 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90012a-30a9-4440-b495-58a30fb215b9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:41.064307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:41.066007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:41.066663 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:41.067136 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] PATH_INFO: `/v3/projects/7134b15c7b8c4764b7fb480b6d081cd1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:41.069778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7134b15c7b8c4764b7fb480b6d081cd1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:41.072962 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:41.076050 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1324/2636] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:11:41.081431 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90012a-30a9-4440-b495-58a30fb215b9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:41.105396 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90012a-30a9-4440-b495-58a30fb215b9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:41.111757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42236d04-3d40-4f95-871d-01ee14979411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:41.112678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42236d04-3d40-4f95-871d-01ee14979411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:41.112964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42236d04-3d40-4f95-871d-01ee14979411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:41.113245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42236d04-3d40-4f95-871d-01ee14979411 admin admin] PATH_INFO: `/v3/projects/7b0b18be8a4b4d04b5d21ecc4efd9ccc/users/10ac5392afff41798bbfc6c632102c9c/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:41.144405 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42236d04-3d40-4f95-871d-01ee14979411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0b18be8a4b4d04b5d21ecc4efd9ccc, user_id=10ac5392afff41798bbfc6c632102c9c, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:41.147526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42236d04-3d40-4f95-871d-01ee14979411 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:41.158632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7134b15c7b8c4764b7fb480b6d081cd1', 'request_id': 'req-4f86254e-ae11-4d80-99ef-2ba491bb280f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:41.167607 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1325/2637] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:41 2022] PUT /identity/v3/projects/7b0b18be8a4b4d04b5d21ecc4efd9ccc/users/10ac5392afff41798bbfc6c632102c9c/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:41.168326 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7134b15c7b8c4764b7fb480b6d081cd1', 'request_id': 'req-4f86254e-ae11-4d80-99ef-2ba491bb280f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:41.182363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7134b15c7b8c4764b7fb480b6d081cd1', 'request_id': 'req-4f86254e-ae11-4d80-99ef-2ba491bb280f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:41.187898 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00306e83-263f-4f19-9fe1-a20738090470 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:41.188181 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00306e83-263f-4f19-9fe1-a20738090470 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:41.188466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00306e83-263f-4f19-9fe1-a20738090470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:41.204644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] The token cache is being invalidate because project 7134b15c7b8c4764b7fb480b6d081cd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:41.204979 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:41.205739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:41.207658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1313/2638] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:41 2022] DELETE /identity/v3/projects/7134b15c7b8c4764b7fb480b6d081cd1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:41.212445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:41.214333 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:41.236015 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-00306e83-263f-4f19-9fe1-a20738090470 None None] MFA Rules not processed for user `10ac5392afff41798bbfc6c632102c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:41.238242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-00306e83-263f-4f19-9fe1-a20738090470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:41.272520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:41.296684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86254e-ae11-4d80-99ef-2ba491bb280f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:41.340956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1326/2639] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:41.358513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00306e83-263f-4f19-9fe1-a20738090470 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:41.359981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-00306e83-263f-4f19-9fe1-a20738090470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:41.363064 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84ccc9-d1bb-4527-9c4a-eed8ba3b4057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:41.363969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84ccc9-d1bb-4527-9c4a-eed8ba3b4057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:41.364261 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84ccc9-d1bb-4527-9c4a-eed8ba3b4057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:41.364525 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84ccc9-d1bb-4527-9c4a-eed8ba3b4057 admin admin] PATH_INFO: `/v3/users/1844a0395afa4011a7698a0358db30eb` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:41.365810 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84ccc9-d1bb-4527-9c4a-eed8ba3b4057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1844a0395afa4011a7698a0358db30eb)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:41.367780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84ccc9-d1bb-4527-9c4a-eed8ba3b4057 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:41.439764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00306e83-263f-4f19-9fe1-a20738090470 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:41.443918 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92f5bc-6125-4cb8-bf38-7abdf52600a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:41.444815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92f5bc-6125-4cb8-bf38-7abdf52600a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:41.445166 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92f5bc-6125-4cb8-bf38-7abdf52600a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:41.445532 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92f5bc-6125-4cb8-bf38-7abdf52600a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:41.448699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0c84ccc9-d1bb-4527-9c4a-eed8ba3b4057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1844a0395afa4011a7698a0358db30eb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:41.450017 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0c84ccc9-d1bb-4527-9c4a-eed8ba3b4057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1844a0395afa4011a7698a0358db30eb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:41.451320 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0c84ccc9-d1bb-4527-9c4a-eed8ba3b4057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1844a0395afa4011a7698a0358db30eb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:41.456381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92f5bc-6125-4cb8-bf38-7abdf52600a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:41.457974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92f5bc-6125-4cb8-bf38-7abdf52600a7 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:41.468432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1327/2640] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:41.473600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0c84ccc9-d1bb-4527-9c4a-eed8ba3b4057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1844a0395afa4011a7698a0358db30eb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:41.497354 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0c84ccc9-d1bb-4527-9c4a-eed8ba3b4057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1844a0395afa4011a7698a0358db30eb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:41.518169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1314/2641] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:41 2022] DELETE /identity/v3/users/1844a0395afa4011a7698a0358db30eb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:41.882112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92f5bc-6125-4cb8-bf38-7abdf52600a7 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:41.889222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:41.896755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92f5bc-6125-4cb8-bf38-7abdf52600a7 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:41.971671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb46bbb-c0b9-4947-82e0-e7cb17741176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:41.972572 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb46bbb-c0b9-4947-82e0-e7cb17741176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:41.972851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb46bbb-c0b9-4947-82e0-e7cb17741176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:41.973130 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb46bbb-c0b9-4947-82e0-e7cb17741176 admin admin] PATH_INFO: `/v3/projects/14f57cdc9c034d31b8a2994b5cb196c0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:41.974534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb46bbb-c0b9-4947-82e0-e7cb17741176 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f57cdc9c034d31b8a2994b5cb196c0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:41.976554 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb46bbb-c0b9-4947-82e0-e7cb17741176 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:42.031622 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1cb46bbb-c0b9-4947-82e0-e7cb17741176 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f57cdc9c034d31b8a2994b5cb196c0', 'request_id': 'req-1cb46bbb-c0b9-4947-82e0-e7cb17741176'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:42.038642 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1cb46bbb-c0b9-4947-82e0-e7cb17741176 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f57cdc9c034d31b8a2994b5cb196c0', 'request_id': 'req-1cb46bbb-c0b9-4947-82e0-e7cb17741176'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:42.049908 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1cb46bbb-c0b9-4947-82e0-e7cb17741176 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f57cdc9c034d31b8a2994b5cb196c0', 'request_id': 'req-1cb46bbb-c0b9-4947-82e0-e7cb17741176'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:42.064277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1cb46bbb-c0b9-4947-82e0-e7cb17741176 admin admin] The token cache is being invalidate because project 14f57cdc9c034d31b8a2994b5cb196c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:42.064722 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1cb46bbb-c0b9-4947-82e0-e7cb17741176 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:42.065451 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1cb46bbb-c0b9-4947-82e0-e7cb17741176 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:42.067165 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1328/2642] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:41 2022] DELETE /identity/v3/projects/14f57cdc9c034d31b8a2994b5cb196c0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:42.072390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360ba22-bb19-4590-9ded-4e9121883a03 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:42.072704 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360ba22-bb19-4590-9ded-4e9121883a03 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:42.072952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360ba22-bb19-4590-9ded-4e9121883a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:42.095081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9360ba22-bb19-4590-9ded-4e9121883a03 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:42.096353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9360ba22-bb19-4590-9ded-4e9121883a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:42.169223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1315/2643] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:42.178268 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee26e0-0884-43ac-88cc-777f63915dc6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:42.178582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee26e0-0884-43ac-88cc-777f63915dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:42.178915 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee26e0-0884-43ac-88cc-777f63915dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:42.200519 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-33ee26e0-0884-43ac-88cc-777f63915dc6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:42.202419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-33ee26e0-0884-43ac-88cc-777f63915dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:42.211854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1329/2644] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:42.216526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360ba22-bb19-4590-9ded-4e9121883a03 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:42.226885 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360ba22-bb19-4590-9ded-4e9121883a03 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:42.233403 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ec529-4867-4772-9e12-d3d81f4fd5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:42.234619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ec529-4867-4772-9e12-d3d81f4fd5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:42.235271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ec529-4867-4772-9e12-d3d81f4fd5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:42.235560 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ec529-4867-4772-9e12-d3d81f4fd5d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:42.236828 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ec529-4867-4772-9e12-d3d81f4fd5d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:42.237162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ec529-4867-4772-9e12-d3d81f4fd5d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:42.239977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ec529-4867-4772-9e12-d3d81f4fd5d4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:42.247550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:42.255049 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1316/2645] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:11:42.261269 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee26e0-0884-43ac-88cc-777f63915dc6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:42.272637 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee26e0-0884-43ac-88cc-777f63915dc6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:42.278593 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc141e9-4870-473d-a694-fcd88841040b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:42.279513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc141e9-4870-473d-a694-fcd88841040b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:42.280000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc141e9-4870-473d-a694-fcd88841040b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:42.280478 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc141e9-4870-473d-a694-fcd88841040b admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:42.281209 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc141e9-4870-473d-a694-fcd88841040b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:42.283353 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc141e9-4870-473d-a694-fcd88841040b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:42.315257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1330/2646] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:42 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:42.320159 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ec529-4867-4772-9e12-d3d81f4fd5d4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:42.330898 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ec529-4867-4772-9e12-d3d81f4fd5d4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:42.396060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003bdbd-7c6f-4b5d-943c-36ac33e0b962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:42.397089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003bdbd-7c6f-4b5d-943c-36ac33e0b962 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:42.397089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003bdbd-7c6f-4b5d-943c-36ac33e0b962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:42.397446 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003bdbd-7c6f-4b5d-943c-36ac33e0b962 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:42.398033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003bdbd-7c6f-4b5d-943c-36ac33e0b962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:42.400103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003bdbd-7c6f-4b5d-943c-36ac33e0b962 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:42.423641 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1317/2647] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:42 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:11:42.428690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc141e9-4870-473d-a694-fcd88841040b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:42.440117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc141e9-4870-473d-a694-fcd88841040b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:42.445420 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205057f3-f316-4cf5-a018-87a94c473fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:42.446212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205057f3-f316-4cf5-a018-87a94c473fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:42.446503 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205057f3-f316-4cf5-a018-87a94c473fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:42.446783 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205057f3-f316-4cf5-a018-87a94c473fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:42.447524 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205057f3-f316-4cf5-a018-87a94c473fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:42.449165 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205057f3-f316-4cf5-a018-87a94c473fc1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:42.461124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1331/2648] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:42.466345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003bdbd-7c6f-4b5d-943c-36ac33e0b962 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:42.480085 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003bdbd-7c6f-4b5d-943c-36ac33e0b962 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:42.486095 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df524893-0923-4acc-8fb1-49e92821c29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:42.486813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df524893-0923-4acc-8fb1-49e92821c29f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:42.487121 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df524893-0923-4acc-8fb1-49e92821c29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:42.487378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df524893-0923-4acc-8fb1-49e92821c29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:42.488120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df524893-0923-4acc-8fb1-49e92821c29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:42.489823 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df524893-0923-4acc-8fb1-49e92821c29f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:42.509920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1318/2649] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:11:42.516465 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205057f3-f316-4cf5-a018-87a94c473fc1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:42.536158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205057f3-f316-4cf5-a018-87a94c473fc1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:42.545452 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7155f61-3861-4bf2-bd76-4fc5038935be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:42.547109 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7155f61-3861-4bf2-bd76-4fc5038935be admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:42.548445 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7155f61-3861-4bf2-bd76-4fc5038935be admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:42.548956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7155f61-3861-4bf2-bd76-4fc5038935be admin admin] PATH_INFO: `/v3/projects/1216d240bbc54c279b1acceb525b64bd/users/b5516ad5effd4fc0b6e3cc6a24241d47/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:42.569765 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7155f61-3861-4bf2-bd76-4fc5038935be admin admin] RBAC: Authorizing `identity:create_grant(project_id=1216d240bbc54c279b1acceb525b64bd, user_id=b5516ad5effd4fc0b6e3cc6a24241d47, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:42.572724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7155f61-3861-4bf2-bd76-4fc5038935be admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:42.606028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1332/2650] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:42 2022] PUT /identity/v3/projects/1216d240bbc54c279b1acceb525b64bd/users/b5516ad5effd4fc0b6e3cc6a24241d47/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:42.628636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a33c1-eff5-4c60-8dcd-c0e919df4e41 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:42.629493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a33c1-eff5-4c60-8dcd-c0e919df4e41 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:42.630072 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a33c1-eff5-4c60-8dcd-c0e919df4e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:42.646979 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7155f61-3861-4bf2-bd76-4fc5038935be admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:42.649065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b7155f61-3861-4bf2-bd76-4fc5038935be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:42.684163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-016a33c1-eff5-4c60-8dcd-c0e919df4e41 None None] MFA Rules not processed for user `b5516ad5effd4fc0b6e3cc6a24241d47`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:42.686003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-016a33c1-eff5-4c60-8dcd-c0e919df4e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:42.691678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:42.747598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7155f61-3861-4bf2-bd76-4fc5038935be admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:42.754597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd5cd0-fa3c-49ef-875b-d5363acdd8d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57dccfe9f5f4bc397f4c470fbf047dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:42.755398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd5cd0-fa3c-49ef-875b-d5363acdd8d4 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:42.755841 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd5cd0-fa3c-49ef-875b-d5363acdd8d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:42.756275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd5cd0-fa3c-49ef-875b-d5363acdd8d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:42.759239 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-71cd5cd0-fa3c-49ef-875b-d5363acdd8d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:42.774587 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd5cd0-fa3c-49ef-875b-d5363acdd8d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:42.776257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd5cd0-fa3c-49ef-875b-d5363acdd8d4 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:42.785608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1319/2651] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:42.791314 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1333/2652] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:42.800734 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a33c1-eff5-4c60-8dcd-c0e919df4e41 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:42.802487 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-016a33c1-eff5-4c60-8dcd-c0e919df4e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:42.895315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a33c1-eff5-4c60-8dcd-c0e919df4e41 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:42.901755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04762f-2695-41f6-8ee4-108ee4cc153b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:42.902682 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04762f-2695-41f6-8ee4-108ee4cc153b service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:42.903125 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04762f-2695-41f6-8ee4-108ee4cc153b service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:42.903832 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04762f-2695-41f6-8ee4-108ee4cc153b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:42.919314 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04762f-2695-41f6-8ee4-108ee4cc153b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:42.921175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04762f-2695-41f6-8ee4-108ee4cc153b service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:42.933107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1320/2653] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:42.969173 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd5cd0-fa3c-49ef-875b-d5363acdd8d4 service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:43.063426 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd5cd0-fa3c-49ef-875b-d5363acdd8d4 service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:43.069297 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:43.071306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:43.071306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:43.071306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] PATH_INFO: `/v3/users/b5516ad5effd4fc0b6e3cc6a24241d47` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:43.072826 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5516ad5effd4fc0b6e3cc6a24241d47)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:43.075051 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:43.152160 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5516ad5effd4fc0b6e3cc6a24241d47'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:43.175117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5516ad5effd4fc0b6e3cc6a24241d47'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:43.187980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5516ad5effd4fc0b6e3cc6a24241d47'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:43.188942 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5516ad5effd4fc0b6e3cc6a24241d47'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:43.209589 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5516ad5effd4fc0b6e3cc6a24241d47'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:43.218069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1334/2654] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:42 2022] DELETE /identity/v3/users/b5516ad5effd4fc0b6e3cc6a24241d47 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:11:43.224469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599778d3-ae2e-4528-9e53-f5df522b6530 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:43.224754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599778d3-ae2e-4528-9e53-f5df522b6530 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:43.225056 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599778d3-ae2e-4528-9e53-f5df522b6530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:43.246727 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-599778d3-ae2e-4528-9e53-f5df522b6530 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:43.249191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-599778d3-ae2e-4528-9e53-f5df522b6530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:43.257035 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:43.338161 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1321/2655] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:43.349690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:43.351350 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:43.403364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:43.427119 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b198a-dce9-402f-be91-497cefce42c6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:43.431317 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72056c9-fe51-4271-9b71-4231300e810a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:43.432143 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72056c9-fe51-4271-9b71-4231300e810a service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:43.432683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72056c9-fe51-4271-9b71-4231300e810a service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:43.433031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72056c9-fe51-4271-9b71-4231300e810a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:43.444733 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72056c9-fe51-4271-9b71-4231300e810a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:43.446285 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72056c9-fe51-4271-9b71-4231300e810a service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:43.457321 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1335/2656] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:11:43.783224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599778d3-ae2e-4528-9e53-f5df522b6530 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:43.785704 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-599778d3-ae2e-4528-9e53-f5df522b6530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:43.804707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599778d3-ae2e-4528-9e53-f5df522b6530 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:43.811352 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:43.813503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:43.814188 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:43.814736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] PATH_INFO: `/v3/projects/1216d240bbc54c279b1acceb525b64bd` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:43.817301 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1216d240bbc54c279b1acceb525b64bd)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:43.824078 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:43.896242 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1216d240bbc54c279b1acceb525b64bd', 'request_id': 'req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:43.902107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1216d240bbc54c279b1acceb525b64bd', 'request_id': 'req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:43.909384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1216d240bbc54c279b1acceb525b64bd', 'request_id': 'req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:43.924660 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] The token cache is being invalidate because project 1216d240bbc54c279b1acceb525b64bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:43.924936 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:43.925738 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:43.927408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1322/2657] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:43 2022] DELETE /identity/v3/projects/1216d240bbc54c279b1acceb525b64bd => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:46.081367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053a44d-38e1-4de5-aa3f-6e56c5f8763d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:46.082147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053a44d-38e1-4de5-aa3f-6e56c5f8763d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:46.082147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053a44d-38e1-4de5-aa3f-6e56c5f8763d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:46.101294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1053a44d-38e1-4de5-aa3f-6e56c5f8763d None None] MFA Rules not processed for user `0224639fe5d24820bdead1bba468ddda`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:46.102598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1053a44d-38e1-4de5-aa3f-6e56c5f8763d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:46.184517 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1336/2658] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:52.572756 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:52.574593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:52.648061 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca910d4-b7f8-4c9b-a29d-a2d9ba770bf4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:52.652768 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:52.653693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:52.653977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:52.654254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] PATH_INFO: `/v3/users/10ac5392afff41798bbfc6c632102c9c` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:52.655789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] RBAC: Authorizing `identity:delete_user(user_id=10ac5392afff41798bbfc6c632102c9c)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:52.657767 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:52.698760 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:52.714577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10ac5392afff41798bbfc6c632102c9c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:52.715804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10ac5392afff41798bbfc6c632102c9c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:52.716440 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10ac5392afff41798bbfc6c632102c9c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:52.728814 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10ac5392afff41798bbfc6c632102c9c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:52.753160 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10ac5392afff41798bbfc6c632102c9c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:52.779568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1323/2659] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:11:52 2022] DELETE /identity/v3/users/10ac5392afff41798bbfc6c632102c9c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:52.788228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136680ca-86c5-4825-adfc-50d340a7cff6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:52.790185 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136680ca-86c5-4825-adfc-50d340a7cff6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:52.790578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136680ca-86c5-4825-adfc-50d340a7cff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:52.812833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-136680ca-86c5-4825-adfc-50d340a7cff6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:52.814372 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-136680ca-86c5-4825-adfc-50d340a7cff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:52.885888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1337/2660] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:52.908255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:52.910167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:52.976846 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f5fbe-e1b2-4a01-93d3-e021ee67562b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:52.980780 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef7baa-d018-4f53-bdf9-67a23d49c5cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:52.981443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef7baa-d018-4f53-bdf9-67a23d49c5cc service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:52.981843 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef7baa-d018-4f53-bdf9-67a23d49c5cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:52.982116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef7baa-d018-4f53-bdf9-67a23d49c5cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:52.992414 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef7baa-d018-4f53-bdf9-67a23d49c5cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:52.993902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef7baa-d018-4f53-bdf9-67a23d49c5cc service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:53.004736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1324/2661] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:11:53.286302 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136680ca-86c5-4825-adfc-50d340a7cff6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:53.287788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-136680ca-86c5-4825-adfc-50d340a7cff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:53.302946 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136680ca-86c5-4825-adfc-50d340a7cff6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:53.309104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc77f4-ccef-44fa-84f4-be329a6fe279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:53.310461 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc77f4-ccef-44fa-84f4-be329a6fe279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:53.310981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc77f4-ccef-44fa-84f4-be329a6fe279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:53.312215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc77f4-ccef-44fa-84f4-be329a6fe279 admin admin] PATH_INFO: `/v3/projects/7b0b18be8a4b4d04b5d21ecc4efd9ccc` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:53.314511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc77f4-ccef-44fa-84f4-be329a6fe279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b0b18be8a4b4d04b5d21ecc4efd9ccc)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:53.318475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc77f4-ccef-44fa-84f4-be329a6fe279 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:53.376277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cabc77f4-ccef-44fa-84f4-be329a6fe279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b0b18be8a4b4d04b5d21ecc4efd9ccc', 'request_id': 'req-cabc77f4-ccef-44fa-84f4-be329a6fe279'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:53.385134 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cabc77f4-ccef-44fa-84f4-be329a6fe279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b0b18be8a4b4d04b5d21ecc4efd9ccc', 'request_id': 'req-cabc77f4-ccef-44fa-84f4-be329a6fe279'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:53.396977 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cabc77f4-ccef-44fa-84f4-be329a6fe279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b0b18be8a4b4d04b5d21ecc4efd9ccc', 'request_id': 'req-cabc77f4-ccef-44fa-84f4-be329a6fe279'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:53.409198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cabc77f4-ccef-44fa-84f4-be329a6fe279 admin admin] The token cache is being invalidate because project 7b0b18be8a4b4d04b5d21ecc4efd9ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:11:53.409487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cabc77f4-ccef-44fa-84f4-be329a6fe279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:53.410165 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cabc77f4-ccef-44fa-84f4-be329a6fe279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:11:53.411737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1338/2662] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:11:53 2022] DELETE /identity/v3/projects/7b0b18be8a4b4d04b5d21ecc4efd9ccc => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:53.416787 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627b069-adc6-475d-8977-9adee679324d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:53.417106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627b069-adc6-475d-8977-9adee679324d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:53.417383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627b069-adc6-475d-8977-9adee679324d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:53.435906 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8627b069-adc6-475d-8977-9adee679324d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:53.437195 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8627b069-adc6-475d-8977-9adee679324d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:53.498284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1325/2663] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:11:53.507533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb803d-4dd3-498d-8bd8-cd477540b987 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:53.507819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb803d-4dd3-498d-8bd8-cd477540b987 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:53.508085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb803d-4dd3-498d-8bd8-cd477540b987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:53.531383 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7cbb803d-4dd3-498d-8bd8-cd477540b987 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:53.532659 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7cbb803d-4dd3-498d-8bd8-cd477540b987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:53.541423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1339/2664] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:53.546060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627b069-adc6-475d-8977-9adee679324d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:53.556070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627b069-adc6-475d-8977-9adee679324d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:53.561655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bcdb0-37d6-441d-8fc3-533e7c078ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:53.562337 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bcdb0-37d6-441d-8fc3-533e7c078ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:53.562637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bcdb0-37d6-441d-8fc3-533e7c078ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:53.562930 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bcdb0-37d6-441d-8fc3-533e7c078ac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:53.563655 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bcdb0-37d6-441d-8fc3-533e7c078ac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:11:53.563959 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bcdb0-37d6-441d-8fc3-533e7c078ac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:53.565540 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bcdb0-37d6-441d-8fc3-533e7c078ac6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:53.588777 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1326/2665] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:11:53.595990 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb803d-4dd3-498d-8bd8-cd477540b987 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:53.600543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:53.606296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb803d-4dd3-498d-8bd8-cd477540b987 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:53.612176 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ebfd5-0a49-4f8c-bd3b-b2c6dba9a650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:53.613000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ebfd5-0a49-4f8c-bd3b-b2c6dba9a650 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:53.613753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ebfd5-0a49-4f8c-bd3b-b2c6dba9a650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:53.614249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ebfd5-0a49-4f8c-bd3b-b2c6dba9a650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:53.615098 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ebfd5-0a49-4f8c-bd3b-b2c6dba9a650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:53.617290 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ebfd5-0a49-4f8c-bd3b-b2c6dba9a650 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:53.644967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1340/2666] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:11:53 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:53.649452 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bcdb0-37d6-441d-8fc3-533e7c078ac6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:53.660080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bcdb0-37d6-441d-8fc3-533e7c078ac6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:53.704382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:11:53.723116 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4acc1-c1bd-4bfc-8068-a1bcd8dc7050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:53.723795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4acc1-c1bd-4bfc-8068-a1bcd8dc7050 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:53.724080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4acc1-c1bd-4bfc-8068-a1bcd8dc7050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:53.724360 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4acc1-c1bd-4bfc-8068-a1bcd8dc7050 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:53.725187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4acc1-c1bd-4bfc-8068-a1bcd8dc7050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:53.727171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4acc1-c1bd-4bfc-8068-a1bcd8dc7050 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:53.750271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1327/2667] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:11:53 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:11:53.754998 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ebfd5-0a49-4f8c-bd3b-b2c6dba9a650 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:53.765680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ebfd5-0a49-4f8c-bd3b-b2c6dba9a650 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:53.771092 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea945a-69fa-4ef7-979a-29292614ce9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:53.771763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea945a-69fa-4ef7-979a-29292614ce9c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:53.772053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea945a-69fa-4ef7-979a-29292614ce9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:53.772323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea945a-69fa-4ef7-979a-29292614ce9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:53.773088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea945a-69fa-4ef7-979a-29292614ce9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:53.774759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea945a-69fa-4ef7-979a-29292614ce9c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:53.785941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1341/2668] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:53.790076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4acc1-c1bd-4bfc-8068-a1bcd8dc7050 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:53.800831 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4acc1-c1bd-4bfc-8068-a1bcd8dc7050 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:53.806279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309a6e7-b4af-47ae-9469-c99f5e489d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:53.806940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309a6e7-b4af-47ae-9469-c99f5e489d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:53.807228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309a6e7-b4af-47ae-9469-c99f5e489d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:53.807493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309a6e7-b4af-47ae-9469-c99f5e489d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:53.808225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309a6e7-b4af-47ae-9469-c99f5e489d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:53.809933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309a6e7-b4af-47ae-9469-c99f5e489d25 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:53.821388 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1328/2669] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:11:53.826019 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea945a-69fa-4ef7-979a-29292614ce9c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:53.839074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea945a-69fa-4ef7-979a-29292614ce9c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:53.844272 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a757f07-9654-41cc-9c96-47b3ddffc9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:53.845152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a757f07-9654-41cc-9c96-47b3ddffc9e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:53.845491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a757f07-9654-41cc-9c96-47b3ddffc9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:53.845805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a757f07-9654-41cc-9c96-47b3ddffc9e0 admin admin] PATH_INFO: `/v3/projects/6feb47b509504545a0d69f45bde62ea8/users/57ab3b5f3746488e8cc8c7c2245533ce/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:53.863502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a757f07-9654-41cc-9c96-47b3ddffc9e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6feb47b509504545a0d69f45bde62ea8, user_id=57ab3b5f3746488e8cc8c7c2245533ce, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:53.866157 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a757f07-9654-41cc-9c96-47b3ddffc9e0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:53.882091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1342/2670] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:11:53 2022] PUT /identity/v3/projects/6feb47b509504545a0d69f45bde62ea8/users/57ab3b5f3746488e8cc8c7c2245533ce/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:11:53.892822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78962a7-e0a4-49bd-95ac-58cce9585828 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:53.893133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78962a7-e0a4-49bd-95ac-58cce9585828 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:53.893415 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78962a7-e0a4-49bd-95ac-58cce9585828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:53.921750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e78962a7-e0a4-49bd-95ac-58cce9585828 None None] MFA Rules not processed for user `57ab3b5f3746488e8cc8c7c2245533ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:11:53.923019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e78962a7-e0a4-49bd-95ac-58cce9585828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:54.003619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1329/2671] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:11:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:11:54.016572 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a757f07-9654-41cc-9c96-47b3ddffc9e0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:11:54.018881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4a757f07-9654-41cc-9c96-47b3ddffc9e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:11:54.105349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a757f07-9654-41cc-9c96-47b3ddffc9e0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:11:54.109702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a361bf-64d4-49ec-bcb6-239fa8306a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:11:54.110377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a361bf-64d4-49ec-bcb6-239fa8306a64 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:11:54.110656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a361bf-64d4-49ec-bcb6-239fa8306a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:11:54.110929 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a361bf-64d4-49ec-bcb6-239fa8306a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:11:54.122957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a361bf-64d4-49ec-bcb6-239fa8306a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:11:54.125437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a361bf-64d4-49ec-bcb6-239fa8306a64 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:11:54.141038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1343/2672] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:11:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:02.093230 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78962a7-e0a4-49bd-95ac-58cce9585828 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:02.095222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e78962a7-e0a4-49bd-95ac-58cce9585828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:02.163818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78962a7-e0a4-49bd-95ac-58cce9585828 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:02.167740 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a349b6e-4c85-430a-83ed-a63b833ea781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:02.168387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a349b6e-4c85-430a-83ed-a63b833ea781 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:02.168708 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a349b6e-4c85-430a-83ed-a63b833ea781 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:02.168938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a349b6e-4c85-430a-83ed-a63b833ea781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:02.171053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2a349b6e-4c85-430a-83ed-a63b833ea781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:02.245281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a349b6e-4c85-430a-83ed-a63b833ea781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:02.247004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a349b6e-4c85-430a-83ed-a63b833ea781 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:02.257430 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1330/2673] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:02.593732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a361bf-64d4-49ec-bcb6-239fa8306a64 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:02.593732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f7a361bf-64d4-49ec-bcb6-239fa8306a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:02.670341 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a361bf-64d4-49ec-bcb6-239fa8306a64 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:02.675731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:02.676726 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:02.677117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:02.677397 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] PATH_INFO: `/v3/users/6fccd472a19c4bd3adf9d9dbecde431a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:02.679352 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fccd472a19c4bd3adf9d9dbecde431a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:02.680686 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:02.739496 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fccd472a19c4bd3adf9d9dbecde431a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:02.742740 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:02.751928 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fccd472a19c4bd3adf9d9dbecde431a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:02.766172 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fccd472a19c4bd3adf9d9dbecde431a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:02.767260 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fccd472a19c4bd3adf9d9dbecde431a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:02.781388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fccd472a19c4bd3adf9d9dbecde431a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:02.787934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1344/2674] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:02 2022] DELETE /identity/v3/users/6fccd472a19c4bd3adf9d9dbecde431a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:02.792782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4c311-7909-47fa-9328-dda3f8a0e917 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:02.793070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4c311-7909-47fa-9328-dda3f8a0e917 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:02.793353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4c311-7909-47fa-9328-dda3f8a0e917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:02.800642 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:02.814920 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-99e4c311-7909-47fa-9328-dda3f8a0e917 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:02.816332 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-99e4c311-7909-47fa-9328-dda3f8a0e917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:02.911726 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1331/2675] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:02.921913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:02.926703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:03.007154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec7509-1265-430d-944b-115057aa8e18 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:03.013419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47a57a-7eee-4f43-bbf4-4a87054c9829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:03.015358 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47a57a-7eee-4f43-bbf4-4a87054c9829 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:03.016420 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47a57a-7eee-4f43-bbf4-4a87054c9829 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:03.016420 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47a57a-7eee-4f43-bbf4-4a87054c9829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:03.030827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47a57a-7eee-4f43-bbf4-4a87054c9829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:03.032967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47a57a-7eee-4f43-bbf4-4a87054c9829 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:03.047526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1345/2676] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:03.926993 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4c311-7909-47fa-9328-dda3f8a0e917 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:03.929594 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-99e4c311-7909-47fa-9328-dda3f8a0e917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:03.956824 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4c311-7909-47fa-9328-dda3f8a0e917 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:03.973171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:03.974165 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:03.975789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:03.975789 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] PATH_INFO: `/v3/projects/118eea41e8ef41b58e5e525b0fa93c5f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:03.977454 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=118eea41e8ef41b58e5e525b0fa93c5f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:03.980021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:04.040870 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '118eea41e8ef41b58e5e525b0fa93c5f', 'request_id': 'req-1d39d380-72df-4816-a72e-1f59851365ae'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.048484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '118eea41e8ef41b58e5e525b0fa93c5f', 'request_id': 'req-1d39d380-72df-4816-a72e-1f59851365ae'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.055856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '118eea41e8ef41b58e5e525b0fa93c5f', 'request_id': 'req-1d39d380-72df-4816-a72e-1f59851365ae'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.074323 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] The token cache is being invalidate because project 118eea41e8ef41b58e5e525b0fa93c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:04.074722 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.075491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.077364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1332/2677] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:03 2022] DELETE /identity/v3/projects/118eea41e8ef41b58e5e525b0fa93c5f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:12:04.082568 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47a57a-7eee-4f43-bbf4-4a87054c9829 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:04.101957 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47a57a-7eee-4f43-bbf4-4a87054c9829 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:04.197197 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dade2b-c898-4651-9622-0f5f5cabbda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:04.198365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dade2b-c898-4651-9622-0f5f5cabbda8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:04.198725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dade2b-c898-4651-9622-0f5f5cabbda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:04.199078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dade2b-c898-4651-9622-0f5f5cabbda8 admin admin] PATH_INFO: `/v3/users/b4d6bddbbcc54b3fbc5ceaef3630b191` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:04.200351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dade2b-c898-4651-9622-0f5f5cabbda8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d6bddbbcc54b3fbc5ceaef3630b191)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:04.202475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dade2b-c898-4651-9622-0f5f5cabbda8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:04.276565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-13dade2b-c898-4651-9622-0f5f5cabbda8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d6bddbbcc54b3fbc5ceaef3630b191'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.288311 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-13dade2b-c898-4651-9622-0f5f5cabbda8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d6bddbbcc54b3fbc5ceaef3630b191'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.296045 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-13dade2b-c898-4651-9622-0f5f5cabbda8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d6bddbbcc54b3fbc5ceaef3630b191'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.296939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-13dade2b-c898-4651-9622-0f5f5cabbda8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d6bddbbcc54b3fbc5ceaef3630b191'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.305096 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:04.311535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-13dade2b-c898-4651-9622-0f5f5cabbda8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d6bddbbcc54b3fbc5ceaef3630b191'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.319523 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1346/2678] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:04 2022] DELETE /identity/v3/users/b4d6bddbbcc54b3fbc5ceaef3630b191 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:04.631111 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:04.633405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:04.647903 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39d380-72df-4816-a72e-1f59851365ae admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:04.708888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:04.709671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:04.710103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:04.710378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1 admin admin] PATH_INFO: `/v3/projects/55c8142e2062474297b44d53f7e92c55` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:04.711584 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c8142e2062474297b44d53f7e92c55)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:04.713504 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:04.751411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:04.769484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c8142e2062474297b44d53f7e92c55', 'request_id': 'req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.775971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c8142e2062474297b44d53f7e92c55', 'request_id': 'req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.783873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c8142e2062474297b44d53f7e92c55', 'request_id': 'req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.799785 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1 admin admin] The token cache is being invalidate because project 55c8142e2062474297b44d53f7e92c55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:04.800407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.801122 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f48cc3c-0658-417b-bf17-f6ffbddd71b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:04.803851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1333/2679] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:04 2022] DELETE /identity/v3/projects/55c8142e2062474297b44d53f7e92c55 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:04.808812 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbac39-8e58-4d2a-bb39-995fb7a989f8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:04.809104 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbac39-8e58-4d2a-bb39-995fb7a989f8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:04.809386 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbac39-8e58-4d2a-bb39-995fb7a989f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:04.829501 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bafbac39-8e58-4d2a-bb39-995fb7a989f8 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:04.831196 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bafbac39-8e58-4d2a-bb39-995fb7a989f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:04.900291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1347/2680] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:04.908563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce3ed0-b4c0-4348-bb89-1302d6f11a78 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:04.908855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce3ed0-b4c0-4348-bb89-1302d6f11a78 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:04.909141 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce3ed0-b4c0-4348-bb89-1302d6f11a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:04.928318 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-adce3ed0-b4c0-4348-bb89-1302d6f11a78 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:04.929580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-adce3ed0-b4c0-4348-bb89-1302d6f11a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:04.942010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1334/2681] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:04.943552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbac39-8e58-4d2a-bb39-995fb7a989f8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:04.954284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbac39-8e58-4d2a-bb39-995fb7a989f8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:04.959470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c71fa-5fb3-4ca0-a2da-21c837fc4b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:04.960683 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c71fa-5fb3-4ca0-a2da-21c837fc4b67 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:04.960961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c71fa-5fb3-4ca0-a2da-21c837fc4b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:04.961305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c71fa-5fb3-4ca0-a2da-21c837fc4b67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:04.962156 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c71fa-5fb3-4ca0-a2da-21c837fc4b67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:12:04.962533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c71fa-5fb3-4ca0-a2da-21c837fc4b67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:04.964294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c71fa-5fb3-4ca0-a2da-21c837fc4b67 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:04.977875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1348/2682] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:12:04.982494 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce3ed0-b4c0-4348-bb89-1302d6f11a78 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:04.992931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce3ed0-b4c0-4348-bb89-1302d6f11a78 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:04.998279 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770fa6c-0977-45cc-b884-5ae9d07bf842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:04.999021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770fa6c-0977-45cc-b884-5ae9d07bf842 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:04.999297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770fa6c-0977-45cc-b884-5ae9d07bf842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:04.999571 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770fa6c-0977-45cc-b884-5ae9d07bf842 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.000286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770fa6c-0977-45cc-b884-5ae9d07bf842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:05.002274 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770fa6c-0977-45cc-b884-5ae9d07bf842 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:05.033260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1335/2683] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:04 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:05.037855 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c71fa-5fb3-4ca0-a2da-21c837fc4b67 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:05.048520 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c71fa-5fb3-4ca0-a2da-21c837fc4b67 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:05.111333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c2251-0bb1-439b-afc4-34cf110f853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:05.111856 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c2251-0bb1-439b-afc4-34cf110f853a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:05.112151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c2251-0bb1-439b-afc4-34cf110f853a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:05.112429 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c2251-0bb1-439b-afc4-34cf110f853a admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.113132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c2251-0bb1-439b-afc4-34cf110f853a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:05.116057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c2251-0bb1-439b-afc4-34cf110f853a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:05.141522 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1349/2684] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:05 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:05.146170 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770fa6c-0977-45cc-b884-5ae9d07bf842 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:05.157908 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770fa6c-0977-45cc-b884-5ae9d07bf842 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:05.163019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fd4d8-360a-4461-b551-28a085d13618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:05.163701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fd4d8-360a-4461-b551-28a085d13618 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:05.163988 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fd4d8-360a-4461-b551-28a085d13618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:05.164269 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fd4d8-360a-4461-b551-28a085d13618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.165002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fd4d8-360a-4461-b551-28a085d13618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:05.166723 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fd4d8-360a-4461-b551-28a085d13618 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:05.179213 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1336/2685] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:05.183476 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c2251-0bb1-439b-afc4-34cf110f853a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:05.193978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c2251-0bb1-439b-afc4-34cf110f853a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:05.198593 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b099777-6aff-4ce9-8e84-48565f5fb139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:05.199263 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b099777-6aff-4ce9-8e84-48565f5fb139 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:05.199542 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b099777-6aff-4ce9-8e84-48565f5fb139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:05.199816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b099777-6aff-4ce9-8e84-48565f5fb139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.200551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b099777-6aff-4ce9-8e84-48565f5fb139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:05.202368 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b099777-6aff-4ce9-8e84-48565f5fb139 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:05.213275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1350/2686] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:05.217989 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fd4d8-360a-4461-b551-28a085d13618 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:05.228483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fd4d8-360a-4461-b551-28a085d13618 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:05.234253 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267854d-29d4-4d36-ad54-ad2ff30398ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:05.235162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267854d-29d4-4d36-ad54-ad2ff30398ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:05.235449 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267854d-29d4-4d36-ad54-ad2ff30398ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:05.235728 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267854d-29d4-4d36-ad54-ad2ff30398ce admin admin] PATH_INFO: `/v3/projects/d996932530b84c6287fe557bacc1329e/users/fe59bed226d64d06a351871389ded3db/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.254905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267854d-29d4-4d36-ad54-ad2ff30398ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d996932530b84c6287fe557bacc1329e, user_id=fe59bed226d64d06a351871389ded3db, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:05.257635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267854d-29d4-4d36-ad54-ad2ff30398ce admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:05.279586 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1337/2687] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:05 2022] PUT /identity/v3/projects/d996932530b84c6287fe557bacc1329e/users/fe59bed226d64d06a351871389ded3db/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:05.291651 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866dce73-f1d7-43ce-b25a-e122fffd921c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:05.291941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866dce73-f1d7-43ce-b25a-e122fffd921c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:05.292305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866dce73-f1d7-43ce-b25a-e122fffd921c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.330966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-866dce73-f1d7-43ce-b25a-e122fffd921c None None] MFA Rules not processed for user `fe59bed226d64d06a351871389ded3db`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:05.331267 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-866dce73-f1d7-43ce-b25a-e122fffd921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:05.386449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:05.415714 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1351/2688] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:05.420570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267854d-29d4-4d36-ad54-ad2ff30398ce admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:05.520643 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267854d-29d4-4d36-ad54-ad2ff30398ce admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:05.526234 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebc04c-7552-4245-ad3f-9bc3d3589206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:05.527162 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebc04c-7552-4245-ad3f-9bc3d3589206 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:05.527639 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebc04c-7552-4245-ad3f-9bc3d3589206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:05.528081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebc04c-7552-4245-ad3f-9bc3d3589206 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.529216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebc04c-7552-4245-ad3f-9bc3d3589206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:05.532082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebc04c-7552-4245-ad3f-9bc3d3589206 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:05.567208 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1338/2689] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:05 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:05.573690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866dce73-f1d7-43ce-b25a-e122fffd921c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:05.586674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866dce73-f1d7-43ce-b25a-e122fffd921c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:05.652340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c7b1e-d07f-4799-8245-bd1dfba08e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:05.653091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c7b1e-d07f-4799-8245-bd1dfba08e1a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:05.653420 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c7b1e-d07f-4799-8245-bd1dfba08e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:05.653804 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c7b1e-d07f-4799-8245-bd1dfba08e1a admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.654550 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c7b1e-d07f-4799-8245-bd1dfba08e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:05.656643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c7b1e-d07f-4799-8245-bd1dfba08e1a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:05.680044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1352/2690] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:05 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:05.683701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebc04c-7552-4245-ad3f-9bc3d3589206 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:05.696075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebc04c-7552-4245-ad3f-9bc3d3589206 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:05.700808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea649db7-e0a6-400e-bc40-98e0ca3fd920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:05.701480 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea649db7-e0a6-400e-bc40-98e0ca3fd920 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:05.702128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea649db7-e0a6-400e-bc40-98e0ca3fd920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:05.702650 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea649db7-e0a6-400e-bc40-98e0ca3fd920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.703416 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea649db7-e0a6-400e-bc40-98e0ca3fd920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:05.705126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea649db7-e0a6-400e-bc40-98e0ca3fd920 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:05.718147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1339/2691] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:12:05.722976 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c7b1e-d07f-4799-8245-bd1dfba08e1a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:05.734129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c7b1e-d07f-4799-8245-bd1dfba08e1a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:05.739570 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a007b-7f58-4017-bec3-f507d3df7a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:05.740507 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a007b-7f58-4017-bec3-f507d3df7a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:05.740787 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a007b-7f58-4017-bec3-f507d3df7a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:05.741113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a007b-7f58-4017-bec3-f507d3df7a1f admin admin] PATH_INFO: `/v3/projects/645ff98880ec4461b7a9a9f55b3f74bd/users/88e2bc3663b548ff9d404e8757e44689/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.759089 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a007b-7f58-4017-bec3-f507d3df7a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=645ff98880ec4461b7a9a9f55b3f74bd, user_id=88e2bc3663b548ff9d404e8757e44689, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:05.763286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a007b-7f58-4017-bec3-f507d3df7a1f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:05.782068 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1353/2692] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:05 2022] PUT /identity/v3/projects/645ff98880ec4461b7a9a9f55b3f74bd/users/88e2bc3663b548ff9d404e8757e44689/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:05.787473 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea649db7-e0a6-400e-bc40-98e0ca3fd920 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:05.800069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea649db7-e0a6-400e-bc40-98e0ca3fd920 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:05.826001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:05.866220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a0a12-7d0e-4be5-bc92-4ca2ecbbdf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:05.867004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a0a12-7d0e-4be5-bc92-4ca2ecbbdf32 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:05.867297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a0a12-7d0e-4be5-bc92-4ca2ecbbdf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:05.867574 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a0a12-7d0e-4be5-bc92-4ca2ecbbdf32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.868317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a0a12-7d0e-4be5-bc92-4ca2ecbbdf32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:05.870031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a0a12-7d0e-4be5-bc92-4ca2ecbbdf32 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:05.890867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1340/2693] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:05.896730 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a007b-7f58-4017-bec3-f507d3df7a1f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:05.912220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a007b-7f58-4017-bec3-f507d3df7a1f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:05.919193 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207f88a-2b59-43b4-b259-ea58827d63cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:05.920219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207f88a-2b59-43b4-b259-ea58827d63cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:05.920530 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207f88a-2b59-43b4-b259-ea58827d63cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:05.920898 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207f88a-2b59-43b4-b259-ea58827d63cb admin admin] PATH_INFO: `/v3/projects/645ff98880ec4461b7a9a9f55b3f74bd/users/88e2bc3663b548ff9d404e8757e44689/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.923761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207f88a-2b59-43b4-b259-ea58827d63cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=645ff98880ec4461b7a9a9f55b3f74bd, user_id=88e2bc3663b548ff9d404e8757e44689, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:05.927708 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207f88a-2b59-43b4-b259-ea58827d63cb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:05.941214 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1354/2694] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:05 2022] PUT /identity/v3/projects/645ff98880ec4461b7a9a9f55b3f74bd/users/88e2bc3663b548ff9d404e8757e44689/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:05.952239 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f338719-a30e-4d7a-903e-603eb96e4cb0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:05.952881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f338719-a30e-4d7a-903e-603eb96e4cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:05.953023 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f338719-a30e-4d7a-903e-603eb96e4cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:05.990245 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5f338719-a30e-4d7a-903e-603eb96e4cb0 None None] MFA Rules not processed for user `88e2bc3663b548ff9d404e8757e44689`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:05.991520 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5f338719-a30e-4d7a-903e-603eb96e4cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:06.067701 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1341/2695] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:06.079252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207f88a-2b59-43b4-b259-ea58827d63cb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:06.081280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5207f88a-2b59-43b4-b259-ea58827d63cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:06.158110 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207f88a-2b59-43b4-b259-ea58827d63cb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:06.162003 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58904924-436f-414f-8e78-182f7d37770c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:06.162762 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58904924-436f-414f-8e78-182f7d37770c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:06.163105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58904924-436f-414f-8e78-182f7d37770c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:06.163443 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58904924-436f-414f-8e78-182f7d37770c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:06.173299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58904924-436f-414f-8e78-182f7d37770c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:06.174854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58904924-436f-414f-8e78-182f7d37770c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:06.185294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1355/2696] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:06.264933 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f338719-a30e-4d7a-903e-603eb96e4cb0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:06.275726 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f338719-a30e-4d7a-903e-603eb96e4cb0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:06.280684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a62d5-40d9-45cc-b9c6-bbb91a1ce257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:06.281353 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a62d5-40d9-45cc-b9c6-bbb91a1ce257 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:06.281679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a62d5-40d9-45cc-b9c6-bbb91a1ce257 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:06.282056 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a62d5-40d9-45cc-b9c6-bbb91a1ce257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:06.357818 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a62d5-40d9-45cc-b9c6-bbb91a1ce257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:06.359799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a62d5-40d9-45cc-b9c6-bbb91a1ce257 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:06.371391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1342/2697] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:08.990094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58904924-436f-414f-8e78-182f7d37770c service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:08.991513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-58904924-436f-414f-8e78-182f7d37770c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:09.037321 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:09.052100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58904924-436f-414f-8e78-182f7d37770c service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:09.056080 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9407a-1132-47a6-9a1b-3d7701a65921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:09.056749 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9407a-1132-47a6-9a1b-3d7701a65921 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:09.057036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9407a-1132-47a6-9a1b-3d7701a65921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:09.057316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9407a-1132-47a6-9a1b-3d7701a65921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:09.059419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b2d9407a-1132-47a6-9a1b-3d7701a65921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:09.133133 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9407a-1132-47a6-9a1b-3d7701a65921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:09.135719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9407a-1132-47a6-9a1b-3d7701a65921 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:09.146374 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1356/2698] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:16.258482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a62d5-40d9-45cc-b9c6-bbb91a1ce257 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:16.260060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-495a62d5-40d9-45cc-b9c6-bbb91a1ce257 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:16.275902 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a62d5-40d9-45cc-b9c6-bbb91a1ce257 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:16.280915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:16.281848 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:16.282189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:16.282436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] PATH_INFO: `/v3/users/a860176540c64e6db269f9ebe4d75753` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:16.283817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=a860176540c64e6db269f9ebe4d75753)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:16.286038 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:16.390077 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:16.395611 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a860176540c64e6db269f9ebe4d75753'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:16.396517 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a860176540c64e6db269f9ebe4d75753'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:16.398967 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a860176540c64e6db269f9ebe4d75753'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:16.414192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a860176540c64e6db269f9ebe4d75753'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:16.445516 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a860176540c64e6db269f9ebe4d75753'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:16.467691 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1343/2699] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:16 2022] DELETE /identity/v3/users/a860176540c64e6db269f9ebe4d75753 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:17.082797 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9407a-1132-47a6-9a1b-3d7701a65921 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:17.099180 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9407a-1132-47a6-9a1b-3d7701a65921 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:17.207543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:17.209422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:17.210024 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:17.210575 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] PATH_INFO: `/v3/projects/2609f0b160e0450798be03f7d3e993ad` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:17.217809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2609f0b160e0450798be03f7d3e993ad)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:17.218537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:17.305381 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2609f0b160e0450798be03f7d3e993ad', 'request_id': 'req-6bd2b06d-f35a-4782-a275-562d6f0943ac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:17.318663 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2609f0b160e0450798be03f7d3e993ad', 'request_id': 'req-6bd2b06d-f35a-4782-a275-562d6f0943ac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:17.332088 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2609f0b160e0450798be03f7d3e993ad', 'request_id': 'req-6bd2b06d-f35a-4782-a275-562d6f0943ac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:17.351852 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] The token cache is being invalidate because project 2609f0b160e0450798be03f7d3e993ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:17.352180 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:17.352995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:17.354676 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1357/2700] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:17 2022] DELETE /identity/v3/projects/2609f0b160e0450798be03f7d3e993ad => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:17.360715 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:17.362849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:17.383001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b580a31-9aaf-4752-b6cd-3e5355e766fe admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:17.461474 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:17.462795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:17.463468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:17.463817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] PATH_INFO: `/v3/users/7b3c639531b14f06ba93f30ee5c7a7c1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:17.465307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3c639531b14f06ba93f30ee5c7a7c1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:17.467441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:17.528378 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3c639531b14f06ba93f30ee5c7a7c1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:17.529168 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3c639531b14f06ba93f30ee5c7a7c1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:17.529846 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3c639531b14f06ba93f30ee5c7a7c1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:17.538872 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3c639531b14f06ba93f30ee5c7a7c1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:17.553696 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3c639531b14f06ba93f30ee5c7a7c1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:17.570679 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1344/2701] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:17 2022] DELETE /identity/v3/users/7b3c639531b14f06ba93f30ee5c7a7c1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:17.797313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:17.798972 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:17.825488 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2b06d-f35a-4782-a275-562d6f0943ac admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:17.845685 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:17.879627 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:17.882985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:17.914217 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:17.915283 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:17.916191 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:17.916473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] PATH_INFO: `/v3/projects/2f59148d214f4b94a9f9787dc1068775` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:17.917790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f59148d214f4b94a9f9787dc1068775)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:17.919728 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:17.949406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:17.981188 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50319dba-3304-4159-bb6b-6ca358b8c790 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:18.002375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f59148d214f4b94a9f9787dc1068775', 'request_id': 'req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.011305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f59148d214f4b94a9f9787dc1068775', 'request_id': 'req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.024563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f59148d214f4b94a9f9787dc1068775', 'request_id': 'req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.046384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] The token cache is being invalidate because project 2f59148d214f4b94a9f9787dc1068775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:18.047284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.049211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.053967 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1358/2702] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:17 2022] DELETE /identity/v3/projects/2f59148d214f4b94a9f9787dc1068775 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:18.101532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f68cd3-bc79-4cb8-aa54-d2384f2ec52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:18.102896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f68cd3-bc79-4cb8-aa54-d2384f2ec52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:18.103186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f68cd3-bc79-4cb8-aa54-d2384f2ec52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:18.103514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f68cd3-bc79-4cb8-aa54-d2384f2ec52c admin admin] PATH_INFO: `/v3/users/57ab3b5f3746488e8cc8c7c2245533ce` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:18.105222 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f68cd3-bc79-4cb8-aa54-d2384f2ec52c admin admin] RBAC: Authorizing `identity:delete_user(user_id=57ab3b5f3746488e8cc8c7c2245533ce)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:18.107742 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f68cd3-bc79-4cb8-aa54-d2384f2ec52c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:18.190655 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:18.195851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-51f68cd3-bc79-4cb8-aa54-d2384f2ec52c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57ab3b5f3746488e8cc8c7c2245533ce'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.195851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-51f68cd3-bc79-4cb8-aa54-d2384f2ec52c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57ab3b5f3746488e8cc8c7c2245533ce'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.201343 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:18.201543 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-51f68cd3-bc79-4cb8-aa54-d2384f2ec52c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57ab3b5f3746488e8cc8c7c2245533ce'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.204995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-51f68cd3-bc79-4cb8-aa54-d2384f2ec52c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57ab3b5f3746488e8cc8c7c2245533ce'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.227238 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d8d13-ed15-4a13-80d7-907f36c64b6f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:18.228171 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-51f68cd3-bc79-4cb8-aa54-d2384f2ec52c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57ab3b5f3746488e8cc8c7c2245533ce'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.236853 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d6f98-cc25-4604-b5ec-fa4a0fa50361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:18.238541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d6f98-cc25-4604-b5ec-fa4a0fa50361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:18.239048 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d6f98-cc25-4604-b5ec-fa4a0fa50361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:18.239761 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d6f98-cc25-4604-b5ec-fa4a0fa50361 admin admin] PATH_INFO: `/v3/users/fe59bed226d64d06a351871389ded3db` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:18.242802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d6f98-cc25-4604-b5ec-fa4a0fa50361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe59bed226d64d06a351871389ded3db)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:18.246722 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d6f98-cc25-4604-b5ec-fa4a0fa50361 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:18.258514 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1345/2703] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:17 2022] DELETE /identity/v3/users/57ab3b5f3746488e8cc8c7c2245533ce => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:18.267391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8aa141-66ec-4b66-8e2c-5084faba9c54 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:18.267867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8aa141-66ec-4b66-8e2c-5084faba9c54 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:18.268369 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8aa141-66ec-4b66-8e2c-5084faba9c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:18.300496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0f8aa141-66ec-4b66-8e2c-5084faba9c54 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:18.302850 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0f8aa141-66ec-4b66-8e2c-5084faba9c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:18.343093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-146d6f98-cc25-4604-b5ec-fa4a0fa50361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe59bed226d64d06a351871389ded3db'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.360320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-146d6f98-cc25-4604-b5ec-fa4a0fa50361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe59bed226d64d06a351871389ded3db'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.373044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-146d6f98-cc25-4604-b5ec-fa4a0fa50361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe59bed226d64d06a351871389ded3db'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.374039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-146d6f98-cc25-4604-b5ec-fa4a0fa50361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe59bed226d64d06a351871389ded3db'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.401012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-146d6f98-cc25-4604-b5ec-fa4a0fa50361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe59bed226d64d06a351871389ded3db'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:18.410848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1359/2704] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:18 2022] DELETE /identity/v3/users/fe59bed226d64d06a351871389ded3db => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:12:18.418176 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ff2ba-991b-4b3b-a61d-89ffcc9dbacb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:18.418546 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ff2ba-991b-4b3b-a61d-89ffcc9dbacb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:18.419049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ff2ba-991b-4b3b-a61d-89ffcc9dbacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:18.419861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1346/2705] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:12:18.434812 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8aa141-66ec-4b66-8e2c-5084faba9c54 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:18.436420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0f8aa141-66ec-4b66-8e2c-5084faba9c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:18.442998 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c99ff2ba-991b-4b3b-a61d-89ffcc9dbacb None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:18.444358 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c99ff2ba-991b-4b3b-a61d-89ffcc9dbacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:18.451156 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:18.466887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:18.525713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8aa141-66ec-4b66-8e2c-5084faba9c54 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:18.527973 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1360/2706] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:12:18.543606 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ff2ba-991b-4b3b-a61d-89ffcc9dbacb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:18.545362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c99ff2ba-991b-4b3b-a61d-89ffcc9dbacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:18.598083 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b73255-e080-4b6b-b7af-f9a4ee4ff362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:18.598896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b73255-e080-4b6b-b7af-f9a4ee4ff362 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:18.599044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b73255-e080-4b6b-b7af-f9a4ee4ff362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:18.599331 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b73255-e080-4b6b-b7af-f9a4ee4ff362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:18.609940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b73255-e080-4b6b-b7af-f9a4ee4ff362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:18.611680 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b73255-e080-4b6b-b7af-f9a4ee4ff362 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:18.623529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1347/2707] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:12:18.625101 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ff2ba-991b-4b3b-a61d-89ffcc9dbacb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:18.629953 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474e261-792b-4229-9dde-4382be006627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:18.630787 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474e261-792b-4229-9dde-4382be006627 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:18.631247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474e261-792b-4229-9dde-4382be006627 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:18.631687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474e261-792b-4229-9dde-4382be006627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:18.643656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474e261-792b-4229-9dde-4382be006627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:18.645266 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474e261-792b-4229-9dde-4382be006627 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:18.657585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1361/2708] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:12:18.994968 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b73255-e080-4b6b-b7af-f9a4ee4ff362 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:18.996741 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-94b73255-e080-4b6b-b7af-f9a4ee4ff362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:19.014292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b73255-e080-4b6b-b7af-f9a4ee4ff362 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:19.021122 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474e261-792b-4229-9dde-4382be006627 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:19.021819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167392a3-5b15-4073-9865-af3b3917f902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:19.023049 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7474e261-792b-4229-9dde-4382be006627 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:19.029763 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167392a3-5b15-4073-9865-af3b3917f902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.031490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167392a3-5b15-4073-9865-af3b3917f902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.032390 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167392a3-5b15-4073-9865-af3b3917f902 admin admin] PATH_INFO: `/v3/projects/6feb47b509504545a0d69f45bde62ea8` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.034670 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167392a3-5b15-4073-9865-af3b3917f902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6feb47b509504545a0d69f45bde62ea8)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:19.037020 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167392a3-5b15-4073-9865-af3b3917f902 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:19.037968 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474e261-792b-4229-9dde-4382be006627 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:19.044095 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:19.045007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.045858 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.046650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] PATH_INFO: `/v3/projects/d996932530b84c6287fe557bacc1329e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.048021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d996932530b84c6287fe557bacc1329e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:19.050678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:19.105937 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-167392a3-5b15-4073-9865-af3b3917f902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6feb47b509504545a0d69f45bde62ea8', 'request_id': 'req-167392a3-5b15-4073-9865-af3b3917f902'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.117549 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-167392a3-5b15-4073-9865-af3b3917f902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6feb47b509504545a0d69f45bde62ea8', 'request_id': 'req-167392a3-5b15-4073-9865-af3b3917f902'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.119803 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd996932530b84c6287fe557bacc1329e', 'request_id': 'req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.127442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd996932530b84c6287fe557bacc1329e', 'request_id': 'req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.128846 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-167392a3-5b15-4073-9865-af3b3917f902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6feb47b509504545a0d69f45bde62ea8', 'request_id': 'req-167392a3-5b15-4073-9865-af3b3917f902'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.140880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd996932530b84c6287fe557bacc1329e', 'request_id': 'req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.150289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-167392a3-5b15-4073-9865-af3b3917f902 admin admin] The token cache is being invalidate because project 6feb47b509504545a0d69f45bde62ea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:19.150708 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-167392a3-5b15-4073-9865-af3b3917f902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.151545 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-167392a3-5b15-4073-9865-af3b3917f902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.153752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1348/2709] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:18 2022] DELETE /identity/v3/projects/6feb47b509504545a0d69f45bde62ea8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:19.159541 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b2ae8-00b0-4711-ae8a-e654a54521a2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.159864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b2ae8-00b0-4711-ae8a-e654a54521a2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.160157 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b2ae8-00b0-4711-ae8a-e654a54521a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.162039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] The token cache is being invalidate because project d996932530b84c6287fe557bacc1329e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:19.162362 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.163002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.164870 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1362/2710] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:19 2022] DELETE /identity/v3/projects/d996932530b84c6287fe557bacc1329e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:19.170412 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:19.172279 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:19.180689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-123b2ae8-00b0-4711-ae8a-e654a54521a2 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:19.182011 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-123b2ae8-00b0-4711-ae8a-e654a54521a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:19.227922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:19.235807 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:19.245081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0a66d-0dfd-4a78-98dd-eaddf939b6c3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:19.251964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:19.253859 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.254212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.254531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] PATH_INFO: `/v3/users/88e2bc3663b548ff9d404e8757e44689` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.255960 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88e2bc3663b548ff9d404e8757e44689)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:19.258402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:19.262434 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1349/2711] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:12:19.285595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d62d8-cf37-484f-b4a5-2a6f327dd2eb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.285595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d62d8-cf37-484f-b4a5-2a6f327dd2eb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.285595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d62d8-cf37-484f-b4a5-2a6f327dd2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.312423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b21d62d8-cf37-484f-b4a5-2a6f327dd2eb None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:19.313705 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b21d62d8-cf37-484f-b4a5-2a6f327dd2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:19.322358 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1350/2712] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:12:19.327494 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d62d8-cf37-484f-b4a5-2a6f327dd2eb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:19.334557 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88e2bc3663b548ff9d404e8757e44689'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.343882 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d62d8-cf37-484f-b4a5-2a6f327dd2eb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:19.349207 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88e2bc3663b548ff9d404e8757e44689'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.350436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3064eab-2222-4ffd-ac1d-b61b11a3b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:19.351183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3064eab-2222-4ffd-ac1d-b61b11a3b9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.351485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3064eab-2222-4ffd-ac1d-b61b11a3b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.351934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3064eab-2222-4ffd-ac1d-b61b11a3b9a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.353666 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3064eab-2222-4ffd-ac1d-b61b11a3b9a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:12:19.353807 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3064eab-2222-4ffd-ac1d-b61b11a3b9a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:19.356753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3064eab-2222-4ffd-ac1d-b61b11a3b9a8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:19.363461 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88e2bc3663b548ff9d404e8757e44689'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.364300 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88e2bc3663b548ff9d404e8757e44689'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.379275 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1351/2713] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:12:19.380365 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3064eab-2222-4ffd-ac1d-b61b11a3b9a8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:19.382028 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88e2bc3663b548ff9d404e8757e44689'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.382238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e3064eab-2222-4ffd-ac1d-b61b11a3b9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:19.392715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1363/2714] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:19 2022] DELETE /identity/v3/users/88e2bc3663b548ff9d404e8757e44689 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:19.401468 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3064eab-2222-4ffd-ac1d-b61b11a3b9a8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:19.493436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768a167-add7-4eb6-a03e-ee26ee241cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:19.495092 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768a167-add7-4eb6-a03e-ee26ee241cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.495400 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768a167-add7-4eb6-a03e-ee26ee241cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.495715 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768a167-add7-4eb6-a03e-ee26ee241cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.496507 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768a167-add7-4eb6-a03e-ee26ee241cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:19.498769 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768a167-add7-4eb6-a03e-ee26ee241cc2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:19.552595 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1352/2715] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:19 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:12:19.556690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:19.575990 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc5cc5-51aa-4d46-829c-51f9c2d7ba16 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:19.665578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee875f00-29bf-4bab-920a-5b031402cd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:19.666308 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee875f00-29bf-4bab-920a-5b031402cd73 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.666724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee875f00-29bf-4bab-920a-5b031402cd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.667010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee875f00-29bf-4bab-920a-5b031402cd73 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.667704 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee875f00-29bf-4bab-920a-5b031402cd73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:19.669694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee875f00-29bf-4bab-920a-5b031402cd73 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:19.694002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1364/2716] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:19 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:19.702183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768a167-add7-4eb6-a03e-ee26ee241cc2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:19.712192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768a167-add7-4eb6-a03e-ee26ee241cc2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:19.717775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94f3b8-908f-45f3-990d-cf306421253c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:19.718577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94f3b8-908f-45f3-990d-cf306421253c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.718854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94f3b8-908f-45f3-990d-cf306421253c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.719117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94f3b8-908f-45f3-990d-cf306421253c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.719912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94f3b8-908f-45f3-990d-cf306421253c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:19.722299 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94f3b8-908f-45f3-990d-cf306421253c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:19.733680 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1353/2717] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:19.739322 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee875f00-29bf-4bab-920a-5b031402cd73 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:19.757679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee875f00-29bf-4bab-920a-5b031402cd73 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:19.763432 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdedbb5-32b9-4d6b-81f8-9fb8870249cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:19.764129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdedbb5-32b9-4d6b-81f8-9fb8870249cb admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.764408 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdedbb5-32b9-4d6b-81f8-9fb8870249cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.764703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdedbb5-32b9-4d6b-81f8-9fb8870249cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.765483 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdedbb5-32b9-4d6b-81f8-9fb8870249cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:19.767252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdedbb5-32b9-4d6b-81f8-9fb8870249cb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:19.776082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94f3b8-908f-45f3-990d-cf306421253c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:19.779772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9c94f3b8-908f-45f3-990d-cf306421253c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:19.793716 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1365/2718] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:19.797504 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdedbb5-32b9-4d6b-81f8-9fb8870249cb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:19.810411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94f3b8-908f-45f3-990d-cf306421253c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:19.815654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdedbb5-32b9-4d6b-81f8-9fb8870249cb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:19.822298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4484757-89c6-4c51-8812-cb31ceabe7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:19.823444 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4484757-89c6-4c51-8812-cb31ceabe7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.823732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4484757-89c6-4c51-8812-cb31ceabe7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.824012 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4484757-89c6-4c51-8812-cb31ceabe7cd admin admin] PATH_INFO: `/v3/projects/e415ff7c83e64638b9d20db8b3dc1f76/users/5ddc18d66ff047a09ce5bdfe4ad95b06/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.825983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f648670-5af9-4929-ae15-59531c1c677c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:19.826915 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f648670-5af9-4929-ae15-59531c1c677c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.827187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f648670-5af9-4929-ae15-59531c1c677c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.827551 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f648670-5af9-4929-ae15-59531c1c677c admin admin] PATH_INFO: `/v3/projects/645ff98880ec4461b7a9a9f55b3f74bd` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.829088 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f648670-5af9-4929-ae15-59531c1c677c admin admin] RBAC: Authorizing `identity:delete_project(project_id=645ff98880ec4461b7a9a9f55b3f74bd)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:19.831082 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f648670-5af9-4929-ae15-59531c1c677c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:19.844881 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4484757-89c6-4c51-8812-cb31ceabe7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e415ff7c83e64638b9d20db8b3dc1f76, user_id=5ddc18d66ff047a09ce5bdfe4ad95b06, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:19.848198 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4484757-89c6-4c51-8812-cb31ceabe7cd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:19.865693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1366/2719] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:19 2022] PUT /identity/v3/projects/e415ff7c83e64638b9d20db8b3dc1f76/users/5ddc18d66ff047a09ce5bdfe4ad95b06/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:12:19.904031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f648670-5af9-4929-ae15-59531c1c677c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '645ff98880ec4461b7a9a9f55b3f74bd', 'request_id': 'req-4f648670-5af9-4929-ae15-59531c1c677c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.912138 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f648670-5af9-4929-ae15-59531c1c677c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '645ff98880ec4461b7a9a9f55b3f74bd', 'request_id': 'req-4f648670-5af9-4929-ae15-59531c1c677c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.922569 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f648670-5af9-4929-ae15-59531c1c677c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '645ff98880ec4461b7a9a9f55b3f74bd', 'request_id': 'req-4f648670-5af9-4929-ae15-59531c1c677c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.939426 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f648670-5af9-4929-ae15-59531c1c677c admin admin] The token cache is being invalidate because project 645ff98880ec4461b7a9a9f55b3f74bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:19.940005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f648670-5af9-4929-ae15-59531c1c677c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.940709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f648670-5af9-4929-ae15-59531c1c677c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:19.942743 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1354/2720] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:19 2022] DELETE /identity/v3/projects/645ff98880ec4461b7a9a9f55b3f74bd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:12:19.949668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88c195-1891-4ef8-900c-3f0893ff6625 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.949668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88c195-1891-4ef8-900c-3f0893ff6625 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.949668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88c195-1891-4ef8-900c-3f0893ff6625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.952853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af27db-93ff-4989-ac3c-d8757366baa6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:19.953192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af27db-93ff-4989-ac3c-d8757366baa6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:19.953672 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af27db-93ff-4989-ac3c-d8757366baa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:19.975029 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-72af27db-93ff-4989-ac3c-d8757366baa6 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:19.976919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-72af27db-93ff-4989-ac3c-d8757366baa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:19.980466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7f88c195-1891-4ef8-900c-3f0893ff6625 None None] MFA Rules not processed for user `5ddc18d66ff047a09ce5bdfe4ad95b06`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:19.981742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7f88c195-1891-4ef8-900c-3f0893ff6625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:19.992664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:20.011765 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:20.072539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1367/2721] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:20.073024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1355/2722] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:20.083753 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c24361-8a14-4cad-9096-1eaa57d51f14 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:20.084261 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c24361-8a14-4cad-9096-1eaa57d51f14 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:20.084529 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c24361-8a14-4cad-9096-1eaa57d51f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:20.087357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af27db-93ff-4989-ac3c-d8757366baa6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:20.089373 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-72af27db-93ff-4989-ac3c-d8757366baa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:20.109031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-90c24361-8a14-4cad-9096-1eaa57d51f14 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:20.111291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-90c24361-8a14-4cad-9096-1eaa57d51f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:20.125123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1368/2723] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:20.132224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c24361-8a14-4cad-9096-1eaa57d51f14 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:20.142724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c24361-8a14-4cad-9096-1eaa57d51f14 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:20.147791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb619531-0b54-4245-a625-d982038b024f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:20.148486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb619531-0b54-4245-a625-d982038b024f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:20.148956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb619531-0b54-4245-a625-d982038b024f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:20.149260 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb619531-0b54-4245-a625-d982038b024f admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:20.149822 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb619531-0b54-4245-a625-d982038b024f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:12:20.150138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb619531-0b54-4245-a625-d982038b024f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:20.151813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb619531-0b54-4245-a625-d982038b024f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:20.170043 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1369/2724] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 08:12:20.175194 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af27db-93ff-4989-ac3c-d8757366baa6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:20.175555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb619531-0b54-4245-a625-d982038b024f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:20.179985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b036dd-1d03-45b4-974b-72385e530330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:20.180736 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b036dd-1d03-45b4-974b-72385e530330 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:20.181027 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b036dd-1d03-45b4-974b-72385e530330 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:20.181308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b036dd-1d03-45b4-974b-72385e530330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:20.188338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb619531-0b54-4245-a625-d982038b024f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:20.193042 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b036dd-1d03-45b4-974b-72385e530330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:20.194118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64f758-8d4d-4668-a83c-caf78361a546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:20.195292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64f758-8d4d-4668-a83c-caf78361a546 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:20.195292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64f758-8d4d-4668-a83c-caf78361a546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:20.195479 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64f758-8d4d-4668-a83c-caf78361a546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:20.196105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64f758-8d4d-4668-a83c-caf78361a546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:20.198046 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64f758-8d4d-4668-a83c-caf78361a546 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:20.199664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b036dd-1d03-45b4-974b-72385e530330 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:20.211905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1356/2725] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:12:20.239103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1370/2726] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:20 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:12:20.244411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b036dd-1d03-45b4-974b-72385e530330 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:20.263555 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b036dd-1d03-45b4-974b-72385e530330 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:20.354868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93609706-2a08-49a3-bc6f-acf58a1e7211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:20.355568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93609706-2a08-49a3-bc6f-acf58a1e7211 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:20.355851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93609706-2a08-49a3-bc6f-acf58a1e7211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:20.356123 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93609706-2a08-49a3-bc6f-acf58a1e7211 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:20.356829 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93609706-2a08-49a3-bc6f-acf58a1e7211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:20.359019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93609706-2a08-49a3-bc6f-acf58a1e7211 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:20.382151 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1357/2727] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:20 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:20.388579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64f758-8d4d-4668-a83c-caf78361a546 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:20.403995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64f758-8d4d-4668-a83c-caf78361a546 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:20.410667 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3382ec4-5667-49f0-ba9a-f059235f4212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:20.411467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3382ec4-5667-49f0-ba9a-f059235f4212 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:20.411799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3382ec4-5667-49f0-ba9a-f059235f4212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:20.412218 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3382ec4-5667-49f0-ba9a-f059235f4212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:20.412941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3382ec4-5667-49f0-ba9a-f059235f4212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:20.414802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3382ec4-5667-49f0-ba9a-f059235f4212 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:20.432301 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1371/2728] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:20.436635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93609706-2a08-49a3-bc6f-acf58a1e7211 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:20.448156 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93609706-2a08-49a3-bc6f-acf58a1e7211 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:20.453346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6671c68-441d-47aa-b98b-34dbe005e9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:20.454066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671c68-441d-47aa-b98b-34dbe005e9ba admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:20.454349 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671c68-441d-47aa-b98b-34dbe005e9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:20.454630 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671c68-441d-47aa-b98b-34dbe005e9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:20.455370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6671c68-441d-47aa-b98b-34dbe005e9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:20.456986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6671c68-441d-47aa-b98b-34dbe005e9ba admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:20.473235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1358/2729] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:20.477272 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3382ec4-5667-49f0-ba9a-f059235f4212 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:20.489594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3382ec4-5667-49f0-ba9a-f059235f4212 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:20.496511 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45648b5-5d58-4675-bdb1-538fbf05a213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:20.497502 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45648b5-5d58-4675-bdb1-538fbf05a213 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:20.497880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45648b5-5d58-4675-bdb1-538fbf05a213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:20.498180 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45648b5-5d58-4675-bdb1-538fbf05a213 admin admin] PATH_INFO: `/v3/projects/a0d098bae071483e808ae51b37772106/users/bcba364ac5b94800a8223f241223616e/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:20.525817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45648b5-5d58-4675-bdb1-538fbf05a213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d098bae071483e808ae51b37772106, user_id=bcba364ac5b94800a8223f241223616e, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:20.525817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45648b5-5d58-4675-bdb1-538fbf05a213 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:20.551121 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1372/2730] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:20 2022] PUT /identity/v3/projects/a0d098bae071483e808ae51b37772106/users/bcba364ac5b94800a8223f241223616e/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:20.553659 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4169f-45eb-476b-af5f-a22024b7b906 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:20.554216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4169f-45eb-476b-af5f-a22024b7b906 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:20.554216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4169f-45eb-476b-af5f-a22024b7b906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:20.603033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-00c4169f-45eb-476b-af5f-a22024b7b906 None None] MFA Rules not processed for user `bcba364ac5b94800a8223f241223616e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:20.604465 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-00c4169f-45eb-476b-af5f-a22024b7b906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:20.649853 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:20.681856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1359/2731] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:20.687039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45648b5-5d58-4675-bdb1-538fbf05a213 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:20.763857 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45648b5-5d58-4675-bdb1-538fbf05a213 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:20.769691 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd5f4-5b28-47d1-b9de-d50f553d883a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:20.770292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd5f4-5b28-47d1-b9de-d50f553d883a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:20.770591 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd5f4-5b28-47d1-b9de-d50f553d883a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:20.770871 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd5f4-5b28-47d1-b9de-d50f553d883a admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:20.771582 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd5f4-5b28-47d1-b9de-d50f553d883a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:20.773832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd5f4-5b28-47d1-b9de-d50f553d883a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:20.810788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1373/2732] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:20 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:12:20.816004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4169f-45eb-476b-af5f-a22024b7b906 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:20.829991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4169f-45eb-476b-af5f-a22024b7b906 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:20.916621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f89d0e-dea3-4cae-bcb8-614e619f3f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:20.917508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f89d0e-dea3-4cae-bcb8-614e619f3f68 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:20.917851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f89d0e-dea3-4cae-bcb8-614e619f3f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:20.918201 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f89d0e-dea3-4cae-bcb8-614e619f3f68 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:20.919107 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f89d0e-dea3-4cae-bcb8-614e619f3f68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:20.922523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f89d0e-dea3-4cae-bcb8-614e619f3f68 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:20.950483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1360/2733] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:20 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:12:20.956007 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd5f4-5b28-47d1-b9de-d50f553d883a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:20.969111 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd5f4-5b28-47d1-b9de-d50f553d883a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:20.974021 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6d8a2-f339-4904-98f3-a110f2bbdd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:20.974702 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6d8a2-f339-4904-98f3-a110f2bbdd80 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:20.974982 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6d8a2-f339-4904-98f3-a110f2bbdd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:20.975254 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6d8a2-f339-4904-98f3-a110f2bbdd80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:20.976064 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6d8a2-f339-4904-98f3-a110f2bbdd80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:20.978303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6d8a2-f339-4904-98f3-a110f2bbdd80 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:20.991731 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1374/2734] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:20.998212 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f89d0e-dea3-4cae-bcb8-614e619f3f68 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:21.012753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f89d0e-dea3-4cae-bcb8-614e619f3f68 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:21.019497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af473def-d023-4a7d-aa15-13f91bc7ba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:21.020711 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af473def-d023-4a7d-aa15-13f91bc7ba93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:21.020985 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af473def-d023-4a7d-aa15-13f91bc7ba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:21.021436 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af473def-d023-4a7d-aa15-13f91bc7ba93 admin admin] PATH_INFO: `/v3/projects/0f8a103818c747d892e0e8d52122f715/users/12004ac3c4f747d58dd2ef4054c09bf0/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:21.047357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af473def-d023-4a7d-aa15-13f91bc7ba93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8a103818c747d892e0e8d52122f715, user_id=12004ac3c4f747d58dd2ef4054c09bf0, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:21.050103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af473def-d023-4a7d-aa15-13f91bc7ba93 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:21.066619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1361/2735] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:20 2022] PUT /identity/v3/projects/0f8a103818c747d892e0e8d52122f715/users/12004ac3c4f747d58dd2ef4054c09bf0/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:21.071552 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6d8a2-f339-4904-98f3-a110f2bbdd80 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:21.087978 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6d8a2-f339-4904-98f3-a110f2bbdd80 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:21.095629 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:21.177382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a05d5f-919c-4805-8ece-35c1996bdaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:21.178097 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a05d5f-919c-4805-8ece-35c1996bdaeb admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:21.178390 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a05d5f-919c-4805-8ece-35c1996bdaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:21.178669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a05d5f-919c-4805-8ece-35c1996bdaeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:21.179462 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a05d5f-919c-4805-8ece-35c1996bdaeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:21.181093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a05d5f-919c-4805-8ece-35c1996bdaeb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:21.193693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1375/2736] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:21.198284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af473def-d023-4a7d-aa15-13f91bc7ba93 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:21.210303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af473def-d023-4a7d-aa15-13f91bc7ba93 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:21.215383 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f203e2-b24e-48dc-92e5-7aed55247e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:21.216303 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f203e2-b24e-48dc-92e5-7aed55247e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:21.216590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f203e2-b24e-48dc-92e5-7aed55247e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:21.216867 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f203e2-b24e-48dc-92e5-7aed55247e4b admin admin] PATH_INFO: `/v3/projects/0f8a103818c747d892e0e8d52122f715/users/12004ac3c4f747d58dd2ef4054c09bf0/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:21.219257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f203e2-b24e-48dc-92e5-7aed55247e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8a103818c747d892e0e8d52122f715, user_id=12004ac3c4f747d58dd2ef4054c09bf0, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:21.222008 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f203e2-b24e-48dc-92e5-7aed55247e4b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:21.235512 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1362/2737] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:21 2022] PUT /identity/v3/projects/0f8a103818c747d892e0e8d52122f715/users/12004ac3c4f747d58dd2ef4054c09bf0/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:21.246555 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a44c55-4aca-4e79-887d-d70cb5ed9b97 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:21.247425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a44c55-4aca-4e79-887d-d70cb5ed9b97 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:21.247425 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a44c55-4aca-4e79-887d-d70cb5ed9b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:21.283325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-16a44c55-4aca-4e79-887d-d70cb5ed9b97 None None] MFA Rules not processed for user `12004ac3c4f747d58dd2ef4054c09bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:21.283325 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-16a44c55-4aca-4e79-887d-d70cb5ed9b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:21.368861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f07269-4bdf-4969-a916-9d372f2aa25e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:21.369384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f07269-4bdf-4969-a916-9d372f2aa25e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:21.369834 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f07269-4bdf-4969-a916-9d372f2aa25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:21.380622 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1376/2738] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:12:21.396814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a44c55-4aca-4e79-887d-d70cb5ed9b97 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:21.398644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b2f07269-4bdf-4969-a916-9d372f2aa25e None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:21.400664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b2f07269-4bdf-4969-a916-9d372f2aa25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:21.466376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a44c55-4aca-4e79-887d-d70cb5ed9b97 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:21.471430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb7a8c-d4b0-4457-b7bc-b3d54f1fcf2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:21.472814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb7a8c-d4b0-4457-b7bc-b3d54f1fcf2a service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:21.473327 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb7a8c-d4b0-4457-b7bc-b3d54f1fcf2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:21.473828 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb7a8c-d4b0-4457-b7bc-b3d54f1fcf2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:21.480417 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1363/2739] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:12:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:12:21.486423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb7a8c-d4b0-4457-b7bc-b3d54f1fcf2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:21.488144 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb7a8c-d4b0-4457-b7bc-b3d54f1fcf2a service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:21.500211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1377/2740] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:21.549469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f07269-4bdf-4969-a916-9d372f2aa25e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:21.551060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b2f07269-4bdf-4969-a916-9d372f2aa25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:21.625192 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb7a8c-d4b0-4457-b7bc-b3d54f1fcf2a service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:21.626106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:21.627578 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-54eb7a8c-d4b0-4457-b7bc-b3d54f1fcf2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:21.648094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb7a8c-d4b0-4457-b7bc-b3d54f1fcf2a service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:21.655348 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2738ebd-4a97-45d5-818f-6d748778369e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:21.656823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2738ebd-4a97-45d5-818f-6d748778369e service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:21.657340 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2738ebd-4a97-45d5-818f-6d748778369e service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:21.657952 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2738ebd-4a97-45d5-818f-6d748778369e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:21.676974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f07269-4bdf-4969-a916-9d372f2aa25e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:21.690465 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831f1b2-ebb3-4010-824f-f882dc430b07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:21.691246 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831f1b2-ebb3-4010-824f-f882dc430b07 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:21.691544 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831f1b2-ebb3-4010-824f-f882dc430b07 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:21.692016 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831f1b2-ebb3-4010-824f-f882dc430b07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:21.704236 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831f1b2-ebb3-4010-824f-f882dc430b07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:21.706550 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831f1b2-ebb3-4010-824f-f882dc430b07 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:21.712417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:21.720482 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1364/2741] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:12:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 08:12:21.746713 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2738ebd-4a97-45d5-818f-6d748778369e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:21.749796 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2738ebd-4a97-45d5-818f-6d748778369e service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:21.777693 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1378/2742] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:21 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:12:23.013184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831f1b2-ebb3-4010-824f-f882dc430b07 service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:23.015380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e831f1b2-ebb3-4010-824f-f882dc430b07 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:23.102333 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831f1b2-ebb3-4010-824f-f882dc430b07 service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:23.114445 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:23.116068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:23.116521 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:23.116965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] PATH_INFO: `/v3/users/5ddc18d66ff047a09ce5bdfe4ad95b06` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:23.122177 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ddc18d66ff047a09ce5bdfe4ad95b06)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:23.124914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:23.215079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ddc18d66ff047a09ce5bdfe4ad95b06'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:23.216284 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ddc18d66ff047a09ce5bdfe4ad95b06'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:23.217004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ddc18d66ff047a09ce5bdfe4ad95b06'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:23.230735 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ddc18d66ff047a09ce5bdfe4ad95b06'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:23.258887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ddc18d66ff047a09ce5bdfe4ad95b06'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:23.284952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1365/2743] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:23 2022] DELETE /identity/v3/users/5ddc18d66ff047a09ce5bdfe4ad95b06 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:23.288053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3df0e-6fec-435c-8759-41f60bf7a137 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:23.288603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3df0e-6fec-435c-8759-41f60bf7a137 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:23.288879 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3df0e-6fec-435c-8759-41f60bf7a137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:23.322035 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-07b3df0e-6fec-435c-8759-41f60bf7a137 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:23.324239 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-07b3df0e-6fec-435c-8759-41f60bf7a137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:23.428463 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1379/2744] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:23.437587 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:23.440143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:23.529983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:23.548895 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb724915-371d-4f4e-b4cb-565cb036be02 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:23.555326 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40098a-e926-4158-8283-e29327cb6099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:23.556147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40098a-e926-4158-8283-e29327cb6099 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:23.556531 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40098a-e926-4158-8283-e29327cb6099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:23.556907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40098a-e926-4158-8283-e29327cb6099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:23.570846 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40098a-e926-4158-8283-e29327cb6099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:23.572637 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40098a-e926-4158-8283-e29327cb6099 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:23.586191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1366/2745] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:24.093851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3df0e-6fec-435c-8759-41f60bf7a137 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:24.095624 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-07b3df0e-6fec-435c-8759-41f60bf7a137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:24.143956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3df0e-6fec-435c-8759-41f60bf7a137 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:24.149879 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6377792-588d-4be0-aa70-443fbacc40ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:24.151280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6377792-588d-4be0-aa70-443fbacc40ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:24.151516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6377792-588d-4be0-aa70-443fbacc40ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:24.151966 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6377792-588d-4be0-aa70-443fbacc40ac admin admin] PATH_INFO: `/v3/projects/e415ff7c83e64638b9d20db8b3dc1f76` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:24.153942 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6377792-588d-4be0-aa70-443fbacc40ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e415ff7c83e64638b9d20db8b3dc1f76)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:24.156090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6377792-588d-4be0-aa70-443fbacc40ac admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:24.229564 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d6377792-588d-4be0-aa70-443fbacc40ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e415ff7c83e64638b9d20db8b3dc1f76', 'request_id': 'req-d6377792-588d-4be0-aa70-443fbacc40ac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:24.235919 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d6377792-588d-4be0-aa70-443fbacc40ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e415ff7c83e64638b9d20db8b3dc1f76', 'request_id': 'req-d6377792-588d-4be0-aa70-443fbacc40ac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:24.246369 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d6377792-588d-4be0-aa70-443fbacc40ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e415ff7c83e64638b9d20db8b3dc1f76', 'request_id': 'req-d6377792-588d-4be0-aa70-443fbacc40ac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:24.262040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d6377792-588d-4be0-aa70-443fbacc40ac admin admin] The token cache is being invalidate because project e415ff7c83e64638b9d20db8b3dc1f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:24.262332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d6377792-588d-4be0-aa70-443fbacc40ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:24.262981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d6377792-588d-4be0-aa70-443fbacc40ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:24.265060 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1380/2746] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:24 2022] DELETE /identity/v3/projects/e415ff7c83e64638b9d20db8b3dc1f76 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:24.271005 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49429e8f-4b6a-4796-a855-57d6b98eee1c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:24.271713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49429e8f-4b6a-4796-a855-57d6b98eee1c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:24.271713 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49429e8f-4b6a-4796-a855-57d6b98eee1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:24.292346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-49429e8f-4b6a-4796-a855-57d6b98eee1c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:24.293684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-49429e8f-4b6a-4796-a855-57d6b98eee1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:24.426135 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1367/2747] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:12:24.439123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e37b-317d-4fb7-b359-6a4b55cedc1e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:24.440299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e37b-317d-4fb7-b359-6a4b55cedc1e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:24.440299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e37b-317d-4fb7-b359-6a4b55cedc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:24.469666 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c9a7e37b-317d-4fb7-b359-6a4b55cedc1e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:24.471185 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c9a7e37b-317d-4fb7-b359-6a4b55cedc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:24.494693 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49429e8f-4b6a-4796-a855-57d6b98eee1c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:24.504897 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1381/2748] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:24.515704 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49429e8f-4b6a-4796-a855-57d6b98eee1c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:24.523788 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9223e-954c-4d59-ab60-c7d73afbbfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:24.524994 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9223e-954c-4d59-ab60-c7d73afbbfef admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:24.525457 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9223e-954c-4d59-ab60-c7d73afbbfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:24.526001 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9223e-954c-4d59-ab60-c7d73afbbfef admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:24.527592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9223e-954c-4d59-ab60-c7d73afbbfef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:12:24.528186 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9223e-954c-4d59-ab60-c7d73afbbfef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:24.531485 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9223e-954c-4d59-ab60-c7d73afbbfef admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:24.557526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1368/2749] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:12:24.565039 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7e37b-317d-4fb7-b359-6a4b55cedc1e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:24.577942 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7e37b-317d-4fb7-b359-6a4b55cedc1e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:24.584179 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe228b-63c9-40b7-a2ab-e1b24582955e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:24.585183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe228b-63c9-40b7-a2ab-e1b24582955e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:24.585829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe228b-63c9-40b7-a2ab-e1b24582955e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:24.586273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe228b-63c9-40b7-a2ab-e1b24582955e admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:24.587334 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe228b-63c9-40b7-a2ab-e1b24582955e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:24.589313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe228b-63c9-40b7-a2ab-e1b24582955e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:24.636251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1382/2750] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:24 2022] POST /identity/v3/projects => generated 399 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:12:24.642596 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9223e-954c-4d59-ab60-c7d73afbbfef admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:24.655166 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9223e-954c-4d59-ab60-c7d73afbbfef admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:24.769662 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6638c-edaf-4a5e-802e-390ad7fa2083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:24.771074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6638c-edaf-4a5e-802e-390ad7fa2083 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:24.771721 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6638c-edaf-4a5e-802e-390ad7fa2083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:24.772345 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6638c-edaf-4a5e-802e-390ad7fa2083 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:24.773784 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6638c-edaf-4a5e-802e-390ad7fa2083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:24.777250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6638c-edaf-4a5e-802e-390ad7fa2083 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:24.822190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1369/2751] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:24 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:24.829264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe228b-63c9-40b7-a2ab-e1b24582955e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:24.851133 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe228b-63c9-40b7-a2ab-e1b24582955e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:24.872955 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4aa88-84de-4e12-a5af-28d0ebda5421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:24.879872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4aa88-84de-4e12-a5af-28d0ebda5421 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:24.880382 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4aa88-84de-4e12-a5af-28d0ebda5421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:24.880896 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4aa88-84de-4e12-a5af-28d0ebda5421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:24.896045 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4aa88-84de-4e12-a5af-28d0ebda5421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:24.898019 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4aa88-84de-4e12-a5af-28d0ebda5421 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:24.922125 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1383/2752] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:24.927984 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6638c-edaf-4a5e-802e-390ad7fa2083 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:24.948918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6638c-edaf-4a5e-802e-390ad7fa2083 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:24.950368 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4aa88-84de-4e12-a5af-28d0ebda5421 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:24.952647 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fdd4aa88-84de-4e12-a5af-28d0ebda5421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:24.952817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9787e-6d5f-4b82-8868-4a17db3376b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:24.953375 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9787e-6d5f-4b82-8868-4a17db3376b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:24.953685 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9787e-6d5f-4b82-8868-4a17db3376b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:24.954474 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9787e-6d5f-4b82-8868-4a17db3376b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:24.954681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9787e-6d5f-4b82-8868-4a17db3376b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:24.957243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9787e-6d5f-4b82-8868-4a17db3376b8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:24.971565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4aa88-84de-4e12-a5af-28d0ebda5421 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:24.974909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1370/2753] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:12:24.977246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8667d25-84c1-4e48-9564-ee63de98300a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:24.978603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8667d25-84c1-4e48-9564-ee63de98300a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:24.979090 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8667d25-84c1-4e48-9564-ee63de98300a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:24.979346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9787e-6d5f-4b82-8868-4a17db3376b8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:24.979837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8667d25-84c1-4e48-9564-ee63de98300a admin admin] PATH_INFO: `/v3/users/bcba364ac5b94800a8223f241223616e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:24.981585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8667d25-84c1-4e48-9564-ee63de98300a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcba364ac5b94800a8223f241223616e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:24.983911 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8667d25-84c1-4e48-9564-ee63de98300a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:24.993805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9787e-6d5f-4b82-8868-4a17db3376b8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:25.001393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ce23f-4843-4a35-8ecf-6f7be638a705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:25.003308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ce23f-4843-4a35-8ecf-6f7be638a705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:25.003913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ce23f-4843-4a35-8ecf-6f7be638a705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:25.004460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ce23f-4843-4a35-8ecf-6f7be638a705 admin admin] PATH_INFO: `/v3/projects/cb70c085efb94ebd90b1f9d5add5d32b/users/d069346c79b74f7d9382893fd6046d19/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:25.006531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:25.032140 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ce23f-4843-4a35-8ecf-6f7be638a705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb70c085efb94ebd90b1f9d5add5d32b, user_id=d069346c79b74f7d9382893fd6046d19, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:25.035431 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ce23f-4843-4a35-8ecf-6f7be638a705 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:25.060059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a8667d25-84c1-4e48-9564-ee63de98300a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcba364ac5b94800a8223f241223616e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:25.070869 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1371/2754] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:24 2022] PUT /identity/v3/projects/cb70c085efb94ebd90b1f9d5add5d32b/users/d069346c79b74f7d9382893fd6046d19/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:25.075518 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73cf77-e49d-4ad8-8e0f-7410cfdee16a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:25.075845 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73cf77-e49d-4ad8-8e0f-7410cfdee16a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:25.076079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73cf77-e49d-4ad8-8e0f-7410cfdee16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:25.086622 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a8667d25-84c1-4e48-9564-ee63de98300a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcba364ac5b94800a8223f241223616e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:25.095449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a8667d25-84c1-4e48-9564-ee63de98300a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcba364ac5b94800a8223f241223616e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:25.096802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a8667d25-84c1-4e48-9564-ee63de98300a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcba364ac5b94800a8223f241223616e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:25.121557 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bd73cf77-e49d-4ad8-8e0f-7410cfdee16a None None] MFA Rules not processed for user `d069346c79b74f7d9382893fd6046d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:25.123999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bd73cf77-e49d-4ad8-8e0f-7410cfdee16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:25.131719 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a8667d25-84c1-4e48-9564-ee63de98300a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcba364ac5b94800a8223f241223616e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:25.141678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1384/2755] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:24 2022] DELETE /identity/v3/users/bcba364ac5b94800a8223f241223616e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:12:25.154657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284e902-61c9-4a3a-af63-195a49d75eca None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:25.155436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284e902-61c9-4a3a-af63-195a49d75eca None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:25.156304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:25.157251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284e902-61c9-4a3a-af63-195a49d75eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:25.188821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2284e902-61c9-4a3a-af63-195a49d75eca None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:25.190222 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2284e902-61c9-4a3a-af63-195a49d75eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:25.268054 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1372/2756] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:25.277948 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73cf77-e49d-4ad8-8e0f-7410cfdee16a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:25.280609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bd73cf77-e49d-4ad8-8e0f-7410cfdee16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:25.287258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1385/2757] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:25.307767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284e902-61c9-4a3a-af63-195a49d75eca None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:25.312116 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2284e902-61c9-4a3a-af63-195a49d75eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:25.420013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73cf77-e49d-4ad8-8e0f-7410cfdee16a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:25.462873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284e902-61c9-4a3a-af63-195a49d75eca None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:25.470121 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218573af-9370-4592-bdea-908d7656050d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:25.471344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218573af-9370-4592-bdea-908d7656050d service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:25.471809 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218573af-9370-4592-bdea-908d7656050d service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:25.472975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218573af-9370-4592-bdea-908d7656050d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:25.495610 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218573af-9370-4592-bdea-908d7656050d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:25.498137 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218573af-9370-4592-bdea-908d7656050d service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:25.517500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce176b6-8cae-45d3-a176-ebea01f95bdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:25.519304 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce176b6-8cae-45d3-a176-ebea01f95bdc service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:25.519619 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce176b6-8cae-45d3-a176-ebea01f95bdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:25.520089 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce176b6-8cae-45d3-a176-ebea01f95bdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:25.526103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1386/2758] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:12:25.632108 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce176b6-8cae-45d3-a176-ebea01f95bdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:25.634255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce176b6-8cae-45d3-a176-ebea01f95bdc service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:25.651443 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1373/2759] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:25.990379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218573af-9370-4592-bdea-908d7656050d service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:25.993259 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-218573af-9370-4592-bdea-908d7656050d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:26.023086 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218573af-9370-4592-bdea-908d7656050d service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:26.031434 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:26.033659 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:26.036724 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:26.038335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] PATH_INFO: `/v3/projects/a0d098bae071483e808ae51b37772106` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:26.040654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d098bae071483e808ae51b37772106)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:26.042941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:26.101601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:26.156121 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d098bae071483e808ae51b37772106', 'request_id': 'req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:26.164129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d098bae071483e808ae51b37772106', 'request_id': 'req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:26.177404 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d098bae071483e808ae51b37772106', 'request_id': 'req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:26.207228 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] The token cache is being invalidate because project a0d098bae071483e808ae51b37772106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:26.207813 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:26.209422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:26.217980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1387/2760] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:25 2022] DELETE /identity/v3/projects/a0d098bae071483e808ae51b37772106 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:26.219975 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce176b6-8cae-45d3-a176-ebea01f95bdc service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:26.236013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce176b6-8cae-45d3-a176-ebea01f95bdc service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:26.271574 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:26.341906 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e9eb2-b08b-4239-b3a5-b66c49c90cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:26.343128 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e9eb2-b08b-4239-b3a5-b66c49c90cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:26.343610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e9eb2-b08b-4239-b3a5-b66c49c90cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:26.344003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e9eb2-b08b-4239-b3a5-b66c49c90cf7 admin admin] PATH_INFO: `/v3/users/12004ac3c4f747d58dd2ef4054c09bf0` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:26.345778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e9eb2-b08b-4239-b3a5-b66c49c90cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12004ac3c4f747d58dd2ef4054c09bf0)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:26.348927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e9eb2-b08b-4239-b3a5-b66c49c90cf7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:26.462863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d8e9eb2-b08b-4239-b3a5-b66c49c90cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12004ac3c4f747d58dd2ef4054c09bf0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:26.463819 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d8e9eb2-b08b-4239-b3a5-b66c49c90cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12004ac3c4f747d58dd2ef4054c09bf0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:26.464614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d8e9eb2-b08b-4239-b3a5-b66c49c90cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12004ac3c4f747d58dd2ef4054c09bf0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:26.478790 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d8e9eb2-b08b-4239-b3a5-b66c49c90cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12004ac3c4f747d58dd2ef4054c09bf0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:26.497205 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d8e9eb2-b08b-4239-b3a5-b66c49c90cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12004ac3c4f747d58dd2ef4054c09bf0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:26.529285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1374/2761] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:26 2022] DELETE /identity/v3/users/12004ac3c4f747d58dd2ef4054c09bf0 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:26.970534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:26.975374 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:27.003356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b33007-6aa9-4ebb-a0a1-5f12576c9df4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:27.183665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9d02e-a217-4dd2-97ed-0555649ce314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:27.185441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9d02e-a217-4dd2-97ed-0555649ce314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:27.186112 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9d02e-a217-4dd2-97ed-0555649ce314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:27.186850 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9d02e-a217-4dd2-97ed-0555649ce314 admin admin] PATH_INFO: `/v3/projects/0f8a103818c747d892e0e8d52122f715` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:27.188699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9d02e-a217-4dd2-97ed-0555649ce314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8a103818c747d892e0e8d52122f715)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:27.190803 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9d02e-a217-4dd2-97ed-0555649ce314 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:27.272873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d8c9d02e-a217-4dd2-97ed-0555649ce314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8a103818c747d892e0e8d52122f715', 'request_id': 'req-d8c9d02e-a217-4dd2-97ed-0555649ce314'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:27.285085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d8c9d02e-a217-4dd2-97ed-0555649ce314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8a103818c747d892e0e8d52122f715', 'request_id': 'req-d8c9d02e-a217-4dd2-97ed-0555649ce314'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:27.295696 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d8c9d02e-a217-4dd2-97ed-0555649ce314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8a103818c747d892e0e8d52122f715', 'request_id': 'req-d8c9d02e-a217-4dd2-97ed-0555649ce314'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:27.323768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d8c9d02e-a217-4dd2-97ed-0555649ce314 admin admin] The token cache is being invalidate because project 0f8a103818c747d892e0e8d52122f715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:27.324282 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d8c9d02e-a217-4dd2-97ed-0555649ce314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:27.325461 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d8c9d02e-a217-4dd2-97ed-0555649ce314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:27.329671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1388/2762] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:26 2022] DELETE /identity/v3/projects/0f8a103818c747d892e0e8d52122f715 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:27.335051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1a5f0-8f0a-4c76-bda2-17896eccdb5e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:27.335775 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1a5f0-8f0a-4c76-bda2-17896eccdb5e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:27.336464 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1a5f0-8f0a-4c76-bda2-17896eccdb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:27.374998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ecd1a5f0-8f0a-4c76-bda2-17896eccdb5e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:27.377529 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ecd1a5f0-8f0a-4c76-bda2-17896eccdb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:27.513739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1375/2763] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:27.632799 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff90bc-4250-41d2-b093-38becdff2e63 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:27.633177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff90bc-4250-41d2-b093-38becdff2e63 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:27.633598 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff90bc-4250-41d2-b093-38becdff2e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:27.666309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e9ff90bc-4250-41d2-b093-38becdff2e63 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:27.667859 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e9ff90bc-4250-41d2-b093-38becdff2e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:27.678154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1389/2764] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:27.684387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1a5f0-8f0a-4c76-bda2-17896eccdb5e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:27.700884 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1a5f0-8f0a-4c76-bda2-17896eccdb5e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:27.710182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98c98c-0843-4f87-bfbb-1b01685aaa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:27.711211 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98c98c-0843-4f87-bfbb-1b01685aaa2a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:27.711599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98c98c-0843-4f87-bfbb-1b01685aaa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:27.711986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98c98c-0843-4f87-bfbb-1b01685aaa2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:27.712919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98c98c-0843-4f87-bfbb-1b01685aaa2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:12:27.713466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98c98c-0843-4f87-bfbb-1b01685aaa2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:27.716220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98c98c-0843-4f87-bfbb-1b01685aaa2a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:27.741684 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1376/2765] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:12:27.744147 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff90bc-4250-41d2-b093-38becdff2e63 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:27.758862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff90bc-4250-41d2-b093-38becdff2e63 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:27.790888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6b6f8-1c1c-459c-9ff8-2a9f7e611fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:27.794338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6b6f8-1c1c-459c-9ff8-2a9f7e611fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:27.795171 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6b6f8-1c1c-459c-9ff8-2a9f7e611fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:27.798467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6b6f8-1c1c-459c-9ff8-2a9f7e611fd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:27.800031 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6b6f8-1c1c-459c-9ff8-2a9f7e611fd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:27.809931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6b6f8-1c1c-459c-9ff8-2a9f7e611fd8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:27.853687 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1390/2766] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:27 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:27.856172 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98c98c-0843-4f87-bfbb-1b01685aaa2a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:27.871927 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98c98c-0843-4f87-bfbb-1b01685aaa2a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:27.915835 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:27.955096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fbf554-e350-4987-82cb-cae2ab94e898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:27.955911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbf554-e350-4987-82cb-cae2ab94e898 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:27.956395 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbf554-e350-4987-82cb-cae2ab94e898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:27.956755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbf554-e350-4987-82cb-cae2ab94e898 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:27.957526 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fbf554-e350-4987-82cb-cae2ab94e898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:27.959648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fbf554-e350-4987-82cb-cae2ab94e898 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:27.992297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1377/2767] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:27 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:27.993665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6b6f8-1c1c-459c-9ff8-2a9f7e611fd8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:28.006275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6b6f8-1c1c-459c-9ff8-2a9f7e611fd8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:28.011636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902269a-05d1-4eca-8032-5f14e4bd6125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:28.012332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902269a-05d1-4eca-8032-5f14e4bd6125 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:28.012612 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902269a-05d1-4eca-8032-5f14e4bd6125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:28.012873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902269a-05d1-4eca-8032-5f14e4bd6125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:28.013609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902269a-05d1-4eca-8032-5f14e4bd6125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:28.015455 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902269a-05d1-4eca-8032-5f14e4bd6125 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:28.032052 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1391/2768] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:28.034152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fbf554-e350-4987-82cb-cae2ab94e898 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:28.052648 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fbf554-e350-4987-82cb-cae2ab94e898 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:28.061184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2590a91-5a72-4592-bae9-3efff6415374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:28.062444 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2590a91-5a72-4592-bae9-3efff6415374 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:28.062894 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2590a91-5a72-4592-bae9-3efff6415374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:28.063380 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2590a91-5a72-4592-bae9-3efff6415374 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:28.064651 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2590a91-5a72-4592-bae9-3efff6415374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:28.068033 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2590a91-5a72-4592-bae9-3efff6415374 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:28.084007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1378/2769] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:28.088303 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902269a-05d1-4eca-8032-5f14e4bd6125 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:28.096034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:28.100760 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902269a-05d1-4eca-8032-5f14e4bd6125 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:28.107817 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75621bc0-5d7a-47d2-9389-9c2c4bbc1d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:28.109203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75621bc0-5d7a-47d2-9389-9c2c4bbc1d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:28.109715 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75621bc0-5d7a-47d2-9389-9c2c4bbc1d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:28.110174 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75621bc0-5d7a-47d2-9389-9c2c4bbc1d8b admin admin] PATH_INFO: `/v3/projects/f6baf666febb42e18525963a8c298110/users/a3437643a8614ba8a85336d6c4ea0d5b/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:28.133887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75621bc0-5d7a-47d2-9389-9c2c4bbc1d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6baf666febb42e18525963a8c298110, user_id=a3437643a8614ba8a85336d6c4ea0d5b, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:28.136605 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75621bc0-5d7a-47d2-9389-9c2c4bbc1d8b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:28.159531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1392/2770] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:28 2022] PUT /identity/v3/projects/f6baf666febb42e18525963a8c298110/users/a3437643a8614ba8a85336d6c4ea0d5b/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:28.174298 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68acd84-ab05-4d01-9af1-43efab51c949 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:28.174644 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68acd84-ab05-4d01-9af1-43efab51c949 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:28.175080 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68acd84-ab05-4d01-9af1-43efab51c949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:28.212154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e68acd84-ab05-4d01-9af1-43efab51c949 None None] MFA Rules not processed for user `a3437643a8614ba8a85336d6c4ea0d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:28.213792 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e68acd84-ab05-4d01-9af1-43efab51c949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:28.309459 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1379/2771] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:28.310848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75621bc0-5d7a-47d2-9389-9c2c4bbc1d8b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:28.397206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75621bc0-5d7a-47d2-9389-9c2c4bbc1d8b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:28.402763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed2e82-b024-4a7d-a48a-1061ddaf24e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:28.403346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed2e82-b024-4a7d-a48a-1061ddaf24e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:28.403623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed2e82-b024-4a7d-a48a-1061ddaf24e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:28.403885 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed2e82-b024-4a7d-a48a-1061ddaf24e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:28.408448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed2e82-b024-4a7d-a48a-1061ddaf24e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:28.408448 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed2e82-b024-4a7d-a48a-1061ddaf24e2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:28.452332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1393/2772] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:28 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:28.454264 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68acd84-ab05-4d01-9af1-43efab51c949 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:28.479090 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68acd84-ab05-4d01-9af1-43efab51c949 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:28.577228 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12529e-9d97-477e-86e4-a728e527c594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:28.578050 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12529e-9d97-477e-86e4-a728e527c594 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:28.578330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12529e-9d97-477e-86e4-a728e527c594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:28.578603 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12529e-9d97-477e-86e4-a728e527c594 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:28.579401 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12529e-9d97-477e-86e4-a728e527c594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:28.581241 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12529e-9d97-477e-86e4-a728e527c594 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:28.604986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1380/2773] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:28 2022] POST /identity/v3/users => generated 299 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:28.609772 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed2e82-b024-4a7d-a48a-1061ddaf24e2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:28.621296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed2e82-b024-4a7d-a48a-1061ddaf24e2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:28.628741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b9696-3f04-4208-8af6-df5dd621dc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:28.629566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b9696-3f04-4208-8af6-df5dd621dc01 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:28.629964 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b9696-3f04-4208-8af6-df5dd621dc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:28.630422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b9696-3f04-4208-8af6-df5dd621dc01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:28.631675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b9696-3f04-4208-8af6-df5dd621dc01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:28.634541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b9696-3f04-4208-8af6-df5dd621dc01 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:28.656695 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1394/2774] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:28.658673 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12529e-9d97-477e-86e4-a728e527c594 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:28.676215 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12529e-9d97-477e-86e4-a728e527c594 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:28.682137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b4026-6838-4212-a5ba-1e6b0f4158c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:28.683065 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b4026-6838-4212-a5ba-1e6b0f4158c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:28.683348 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b4026-6838-4212-a5ba-1e6b0f4158c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:28.683632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b4026-6838-4212-a5ba-1e6b0f4158c6 admin admin] PATH_INFO: `/v3/projects/c85cbb3d233347bca0a0fe3596427ce5/users/4c1619def6bd4db8870408855e459199/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:28.702935 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b4026-6838-4212-a5ba-1e6b0f4158c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c85cbb3d233347bca0a0fe3596427ce5, user_id=4c1619def6bd4db8870408855e459199, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:28.706120 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b4026-6838-4212-a5ba-1e6b0f4158c6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:28.724582 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1381/2775] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:28 2022] PUT /identity/v3/projects/c85cbb3d233347bca0a0fe3596427ce5/users/4c1619def6bd4db8870408855e459199/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:28.728741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b9696-3f04-4208-8af6-df5dd621dc01 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:28.741309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b9696-3f04-4208-8af6-df5dd621dc01 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:28.812057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a5fc5-e59d-48de-a85d-3ff3f6fb6430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:28.812727 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a5fc5-e59d-48de-a85d-3ff3f6fb6430 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:28.813015 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a5fc5-e59d-48de-a85d-3ff3f6fb6430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:28.813292 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a5fc5-e59d-48de-a85d-3ff3f6fb6430 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:28.814066 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a5fc5-e59d-48de-a85d-3ff3f6fb6430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:28.819924 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a5fc5-e59d-48de-a85d-3ff3f6fb6430 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:28.834466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1395/2776] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:28.838440 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b4026-6838-4212-a5ba-1e6b0f4158c6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:28.852117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b4026-6838-4212-a5ba-1e6b0f4158c6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:28.857868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d3716-3006-408f-83d3-1a5f1e8eac21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:28.858851 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3716-3006-408f-83d3-1a5f1e8eac21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:28.859127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3716-3006-408f-83d3-1a5f1e8eac21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:28.859402 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3716-3006-408f-83d3-1a5f1e8eac21 admin admin] PATH_INFO: `/v3/projects/c85cbb3d233347bca0a0fe3596427ce5/users/4c1619def6bd4db8870408855e459199/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:28.861855 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d3716-3006-408f-83d3-1a5f1e8eac21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c85cbb3d233347bca0a0fe3596427ce5, user_id=4c1619def6bd4db8870408855e459199, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:28.864759 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d3716-3006-408f-83d3-1a5f1e8eac21 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:28.871182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:28.882871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1382/2777] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:28 2022] PUT /identity/v3/projects/c85cbb3d233347bca0a0fe3596427ce5/users/4c1619def6bd4db8870408855e459199/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:28.894563 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823f9e3-146d-4287-a021-116be8f65b0a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:28.894861 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823f9e3-146d-4287-a021-116be8f65b0a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:28.895152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823f9e3-146d-4287-a021-116be8f65b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:28.931665 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9823f9e3-146d-4287-a021-116be8f65b0a None None] MFA Rules not processed for user `4c1619def6bd4db8870408855e459199`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:28.933513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9823f9e3-146d-4287-a021-116be8f65b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:28.984537 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:29.019860 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1396/2778] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:29.032316 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d3716-3006-408f-83d3-1a5f1e8eac21 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:29.034803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d33d3716-3006-408f-83d3-1a5f1e8eac21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:29.116111 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d3716-3006-408f-83d3-1a5f1e8eac21 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:29.122661 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a48495-f760-4c52-b59a-206b61b5cbdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:29.126101 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48495-f760-4c52-b59a-206b61b5cbdc service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:29.126493 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48495-f760-4c52-b59a-206b61b5cbdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:29.126873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48495-f760-4c52-b59a-206b61b5cbdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:29.144535 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a48495-f760-4c52-b59a-206b61b5cbdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:29.146190 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a48495-f760-4c52-b59a-206b61b5cbdc service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:29.159295 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1383/2779] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:29.191623 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823f9e3-146d-4287-a021-116be8f65b0a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:29.267391 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823f9e3-146d-4287-a021-116be8f65b0a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:29.272681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:29.275013 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:29.275306 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:29.276344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] PATH_INFO: `/v3/users/a3437643a8614ba8a85336d6c4ea0d5b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:29.278117 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3437643a8614ba8a85336d6c4ea0d5b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:29.280152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:29.343974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3437643a8614ba8a85336d6c4ea0d5b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:29.354291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3437643a8614ba8a85336d6c4ea0d5b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:29.366541 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3437643a8614ba8a85336d6c4ea0d5b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:29.367273 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3437643a8614ba8a85336d6c4ea0d5b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:29.384378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3437643a8614ba8a85336d6c4ea0d5b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:29.392802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1397/2780] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:29 2022] DELETE /identity/v3/users/a3437643a8614ba8a85336d6c4ea0d5b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:29.399282 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56228-8a03-4b7c-9706-f48dd5f8301f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:29.399572 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56228-8a03-4b7c-9706-f48dd5f8301f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:29.400247 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56228-8a03-4b7c-9706-f48dd5f8301f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:29.420942 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cce56228-8a03-4b7c-9706-f48dd5f8301f None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:29.424032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cce56228-8a03-4b7c-9706-f48dd5f8301f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:29.502133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1384/2781] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:29.511389 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:29.514320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:29.564508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce56228-8a03-4b7c-9706-f48dd5f8301f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:29.566025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cce56228-8a03-4b7c-9706-f48dd5f8301f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:29.603458 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b9d0c-3375-435d-bd07-8fea1a66a9a1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:29.615042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d99d9f-f335-4531-8a11-c2274e5d35e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:29.615716 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d99d9f-f335-4531-8a11-c2274e5d35e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:29.615995 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d99d9f-f335-4531-8a11-c2274e5d35e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:29.616274 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d99d9f-f335-4531-8a11-c2274e5d35e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:29.629335 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d99d9f-f335-4531-8a11-c2274e5d35e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:29.631917 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d99d9f-f335-4531-8a11-c2274e5d35e7 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:29.636396 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:29.655225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1398/2782] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:29.669068 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce56228-8a03-4b7c-9706-f48dd5f8301f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:29.675191 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393741a-961b-4509-ab0b-87ef0484036a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:29.675995 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393741a-961b-4509-ab0b-87ef0484036a service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:29.676347 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393741a-961b-4509-ab0b-87ef0484036a service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:29.676654 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393741a-961b-4509-ab0b-87ef0484036a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:29.678705 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3393741a-961b-4509-ab0b-87ef0484036a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:29.746868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:29.753597 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393741a-961b-4509-ab0b-87ef0484036a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:29.755940 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393741a-961b-4509-ab0b-87ef0484036a service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:29.769914 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1385/2783] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:12:30.042472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d99d9f-f335-4531-8a11-c2274e5d35e7 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:30.044235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-07d99d9f-f335-4531-8a11-c2274e5d35e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:30.066240 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d99d9f-f335-4531-8a11-c2274e5d35e7 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:30.070921 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:30.072357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:30.072656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:30.072935 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] PATH_INFO: `/v3/projects/f6baf666febb42e18525963a8c298110` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:30.074469 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6baf666febb42e18525963a8c298110)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:30.076476 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:30.146988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6baf666febb42e18525963a8c298110', 'request_id': 'req-2c415e23-fa6a-450f-8794-8771f8df6726'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.153754 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6baf666febb42e18525963a8c298110', 'request_id': 'req-2c415e23-fa6a-450f-8794-8771f8df6726'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.164745 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6baf666febb42e18525963a8c298110', 'request_id': 'req-2c415e23-fa6a-450f-8794-8771f8df6726'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.178833 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] The token cache is being invalidate because project f6baf666febb42e18525963a8c298110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:30.179219 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.180294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.184135 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1399/2784] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:30 2022] DELETE /identity/v3/projects/f6baf666febb42e18525963a8c298110 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:30.188803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393741a-961b-4509-ab0b-87ef0484036a service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:30.201351 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393741a-961b-4509-ab0b-87ef0484036a service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:30.273639 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459386c-9aa2-41ec-929d-e61addd1bfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:30.274816 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459386c-9aa2-41ec-929d-e61addd1bfe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:30.275179 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459386c-9aa2-41ec-929d-e61addd1bfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:30.276342 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459386c-9aa2-41ec-929d-e61addd1bfe5 admin admin] PATH_INFO: `/v3/users/4c1619def6bd4db8870408855e459199` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:30.277944 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459386c-9aa2-41ec-929d-e61addd1bfe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c1619def6bd4db8870408855e459199)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:30.280044 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459386c-9aa2-41ec-929d-e61addd1bfe5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:30.348010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0459386c-9aa2-41ec-929d-e61addd1bfe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c1619def6bd4db8870408855e459199'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.348984 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0459386c-9aa2-41ec-929d-e61addd1bfe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c1619def6bd4db8870408855e459199'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.349707 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0459386c-9aa2-41ec-929d-e61addd1bfe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c1619def6bd4db8870408855e459199'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.357573 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0459386c-9aa2-41ec-929d-e61addd1bfe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c1619def6bd4db8870408855e459199'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.373423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0459386c-9aa2-41ec-929d-e61addd1bfe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c1619def6bd4db8870408855e459199'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.393722 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1386/2785] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:30 2022] DELETE /identity/v3/users/4c1619def6bd4db8870408855e459199 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:30.727154 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:30.728622 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:30.747551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415e23-fa6a-450f-8794-8771f8df6726 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:30.838913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:30.839824 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:30.840177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:30.840477 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0 admin admin] PATH_INFO: `/v3/projects/c85cbb3d233347bca0a0fe3596427ce5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:30.841798 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c85cbb3d233347bca0a0fe3596427ce5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:30.844931 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:30.914376 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c85cbb3d233347bca0a0fe3596427ce5', 'request_id': 'req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.925182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c85cbb3d233347bca0a0fe3596427ce5', 'request_id': 'req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.946789 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c85cbb3d233347bca0a0fe3596427ce5', 'request_id': 'req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.959559 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0 admin admin] The token cache is being invalidate because project c85cbb3d233347bca0a0fe3596427ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:30.959857 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.961579 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4e7457a3-7eb1-44ea-845c-f4a6bdad18b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:30.964706 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1400/2786] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:30 2022] DELETE /identity/v3/projects/c85cbb3d233347bca0a0fe3596427ce5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:30.970888 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac901bf-7187-4e4c-9ed5-423fa072b68c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:30.972045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac901bf-7187-4e4c-9ed5-423fa072b68c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:30.972389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac901bf-7187-4e4c-9ed5-423fa072b68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.009715 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3ac901bf-7187-4e4c-9ed5-423fa072b68c None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:31.015127 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3ac901bf-7187-4e4c-9ed5-423fa072b68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:31.147974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1387/2787] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:12:31.157460 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ea8d2-cae9-4503-b714-ac4bfc64db7e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:31.158069 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ea8d2-cae9-4503-b714-ac4bfc64db7e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:31.158200 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ea8d2-cae9-4503-b714-ac4bfc64db7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.178741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-581ea8d2-cae9-4503-b714-ac4bfc64db7e None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:31.180277 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-581ea8d2-cae9-4503-b714-ac4bfc64db7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:31.189466 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1401/2788] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:12:31.193834 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac901bf-7187-4e4c-9ed5-423fa072b68c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:31.204483 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac901bf-7187-4e4c-9ed5-423fa072b68c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:31.209293 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcf66d-5afc-461d-bcdd-6baecef8cab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:31.209991 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcf66d-5afc-461d-bcdd-6baecef8cab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:31.210271 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcf66d-5afc-461d-bcdd-6baecef8cab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:31.210535 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcf66d-5afc-461d-bcdd-6baecef8cab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.211292 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcf66d-5afc-461d-bcdd-6baecef8cab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:12:31.211592 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcf66d-5afc-461d-bcdd-6baecef8cab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:31.213220 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcf66d-5afc-461d-bcdd-6baecef8cab1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:31.223238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:31.226854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1388/2789] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:12:31.231863 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ea8d2-cae9-4503-b714-ac4bfc64db7e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:31.237975 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:31.241807 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ea8d2-cae9-4503-b714-ac4bfc64db7e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:31.248082 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d7b74-f6ae-4a5d-b4cd-4d47c0d61398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:31.248747 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d7b74-f6ae-4a5d-b4cd-4d47c0d61398 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:31.249338 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d7b74-f6ae-4a5d-b4cd-4d47c0d61398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:31.249800 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d7b74-f6ae-4a5d-b4cd-4d47c0d61398 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.250717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d7b74-f6ae-4a5d-b4cd-4d47c0d61398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:31.252680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d7b74-f6ae-4a5d-b4cd-4d47c0d61398 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:31.280802 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1402/2790] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:31 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:31.285235 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcf66d-5afc-461d-bcdd-6baecef8cab1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:31.296050 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcf66d-5afc-461d-bcdd-6baecef8cab1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:31.355658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde31d6c-1f85-4b48-b37d-6c7840281823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:31.356341 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde31d6c-1f85-4b48-b37d-6c7840281823 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:31.356629 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde31d6c-1f85-4b48-b37d-6c7840281823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:31.356901 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde31d6c-1f85-4b48-b37d-6c7840281823 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.357647 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde31d6c-1f85-4b48-b37d-6c7840281823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:31.359635 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde31d6c-1f85-4b48-b37d-6c7840281823 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:31.382460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1389/2791] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:31 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:12:31.387025 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d7b74-f6ae-4a5d-b4cd-4d47c0d61398 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:31.399361 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d7b74-f6ae-4a5d-b4cd-4d47c0d61398 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:31.404449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3b451-b5f0-421b-8e2b-9b22d18d6b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:31.405208 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3b451-b5f0-421b-8e2b-9b22d18d6b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:31.405650 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3b451-b5f0-421b-8e2b-9b22d18d6b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:31.406215 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3b451-b5f0-421b-8e2b-9b22d18d6b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.406961 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3b451-b5f0-421b-8e2b-9b22d18d6b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:31.408600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3b451-b5f0-421b-8e2b-9b22d18d6b5d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:31.419319 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1403/2792] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:12:31.425877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde31d6c-1f85-4b48-b37d-6c7840281823 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:31.442015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde31d6c-1f85-4b48-b37d-6c7840281823 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:31.450652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1933a2b-79dc-43b9-8e9f-49a62a6ebc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:31.451628 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1933a2b-79dc-43b9-8e9f-49a62a6ebc8c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:31.452059 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1933a2b-79dc-43b9-8e9f-49a62a6ebc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:31.453260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1933a2b-79dc-43b9-8e9f-49a62a6ebc8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.454112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1933a2b-79dc-43b9-8e9f-49a62a6ebc8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:31.456579 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1933a2b-79dc-43b9-8e9f-49a62a6ebc8c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:31.472338 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1390/2793] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:31.477471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3b451-b5f0-421b-8e2b-9b22d18d6b5d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:31.493944 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3b451-b5f0-421b-8e2b-9b22d18d6b5d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:31.501365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d0200-839e-43d8-8f16-5d0207bd67a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:31.502388 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d0200-839e-43d8-8f16-5d0207bd67a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:31.502671 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d0200-839e-43d8-8f16-5d0207bd67a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:31.503280 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d0200-839e-43d8-8f16-5d0207bd67a8 admin admin] PATH_INFO: `/v3/projects/eb455367cec9457c949956902e6494a0/users/c3df9fab59bc4679a8ed5a808adb764f/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.519078 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d0200-839e-43d8-8f16-5d0207bd67a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb455367cec9457c949956902e6494a0, user_id=c3df9fab59bc4679a8ed5a808adb764f, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:31.521796 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d0200-839e-43d8-8f16-5d0207bd67a8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:31.542974 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1404/2794] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:31 2022] PUT /identity/v3/projects/eb455367cec9457c949956902e6494a0/users/c3df9fab59bc4679a8ed5a808adb764f/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:31.559717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b373c-d9e3-439c-b21a-9eea6e3b860a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:31.560081 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b373c-d9e3-439c-b21a-9eea6e3b860a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:31.560471 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b373c-d9e3-439c-b21a-9eea6e3b860a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.590232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-695b373c-d9e3-439c-b21a-9eea6e3b860a None None] MFA Rules not processed for user `c3df9fab59bc4679a8ed5a808adb764f`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:31.591477 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-695b373c-d9e3-439c-b21a-9eea6e3b860a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:31.658387 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1391/2795] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:31.663419 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d0200-839e-43d8-8f16-5d0207bd67a8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:31.741791 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d0200-839e-43d8-8f16-5d0207bd67a8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:31.748212 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87267d-6293-49fd-beda-912dcc769907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:31.749111 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87267d-6293-49fd-beda-912dcc769907 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:31.749423 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87267d-6293-49fd-beda-912dcc769907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:31.749940 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87267d-6293-49fd-beda-912dcc769907 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.750669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87267d-6293-49fd-beda-912dcc769907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:31.752636 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87267d-6293-49fd-beda-912dcc769907 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:31.783668 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1405/2796] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:31 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:31.787856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b373c-d9e3-439c-b21a-9eea6e3b860a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:31.800074 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b373c-d9e3-439c-b21a-9eea6e3b860a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:31.858877 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c5d8a-45a0-4b64-aa4f-f890e63c5929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:31.859553 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c5d8a-45a0-4b64-aa4f-f890e63c5929 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:31.859838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c5d8a-45a0-4b64-aa4f-f890e63c5929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:31.860117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c5d8a-45a0-4b64-aa4f-f890e63c5929 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.860813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c5d8a-45a0-4b64-aa4f-f890e63c5929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:31.862791 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c5d8a-45a0-4b64-aa4f-f890e63c5929 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:31.880024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:31.886616 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1392/2797] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:31 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:31.891363 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87267d-6293-49fd-beda-912dcc769907 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:31.902375 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87267d-6293-49fd-beda-912dcc769907 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:31.908145 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f08a78-47b2-47c1-be97-351ff68d0a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:31.909805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f08a78-47b2-47c1-be97-351ff68d0a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:31.909805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f08a78-47b2-47c1-be97-351ff68d0a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:31.909805 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f08a78-47b2-47c1-be97-351ff68d0a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.910195 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f08a78-47b2-47c1-be97-351ff68d0a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:31.911875 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f08a78-47b2-47c1-be97-351ff68d0a2c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:31.925384 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1406/2798] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 08:12:31.929681 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c5d8a-45a0-4b64-aa4f-f890e63c5929 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:31.940496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c5d8a-45a0-4b64-aa4f-f890e63c5929 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:31.945262 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f86dfd-0251-4c4a-b170-a05de40bb0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:31.946189 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f86dfd-0251-4c4a-b170-a05de40bb0f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:31.946474 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f86dfd-0251-4c4a-b170-a05de40bb0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:31.946752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f86dfd-0251-4c4a-b170-a05de40bb0f5 admin admin] PATH_INFO: `/v3/projects/afaaaf5951b84450b0fe4fac848ced8f/users/9d372256c4844f7cb5affbebbf2571c3/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:31.964474 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f86dfd-0251-4c4a-b170-a05de40bb0f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afaaaf5951b84450b0fe4fac848ced8f, user_id=9d372256c4844f7cb5affbebbf2571c3, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:31.967324 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f86dfd-0251-4c4a-b170-a05de40bb0f5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:31.987133 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1393/2799] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:31 2022] PUT /identity/v3/projects/afaaaf5951b84450b0fe4fac848ced8f/users/9d372256c4844f7cb5affbebbf2571c3/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:31.991832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f08a78-47b2-47c1-be97-351ff68d0a2c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:32.003352 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f08a78-47b2-47c1-be97-351ff68d0a2c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:32.067316 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831129f-3cf1-4760-9fc0-0690f3a1c08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:32.068246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831129f-3cf1-4760-9fc0-0690f3a1c08f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:32.068533 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831129f-3cf1-4760-9fc0-0690f3a1c08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:32.068822 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831129f-3cf1-4760-9fc0-0690f3a1c08f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:32.069556 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831129f-3cf1-4760-9fc0-0690f3a1c08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:32.071236 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831129f-3cf1-4760-9fc0-0690f3a1c08f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:32.082435 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1407/2800] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:32.087069 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f86dfd-0251-4c4a-b170-a05de40bb0f5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:32.098476 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f86dfd-0251-4c4a-b170-a05de40bb0f5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:32.117409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8498b8c-5f5f-48cb-8639-a8313b74193f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:32.117409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8498b8c-5f5f-48cb-8639-a8313b74193f admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:32.117409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8498b8c-5f5f-48cb-8639-a8313b74193f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:32.117409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8498b8c-5f5f-48cb-8639-a8313b74193f admin admin] PATH_INFO: `/v3/projects/afaaaf5951b84450b0fe4fac848ced8f/users/9d372256c4844f7cb5affbebbf2571c3/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:32.118499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8498b8c-5f5f-48cb-8639-a8313b74193f admin admin] RBAC: Authorizing `identity:create_grant(project_id=afaaaf5951b84450b0fe4fac848ced8f, user_id=9d372256c4844f7cb5affbebbf2571c3, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:32.118887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8498b8c-5f5f-48cb-8639-a8313b74193f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:32.134461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1394/2801] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:32 2022] PUT /identity/v3/projects/afaaaf5951b84450b0fe4fac848ced8f/users/9d372256c4844f7cb5affbebbf2571c3/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:32.146182 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f6ed2-0980-4aa0-9a85-4d4eeccf93ce None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:32.146531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f6ed2-0980-4aa0-9a85-4d4eeccf93ce None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:32.146780 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f6ed2-0980-4aa0-9a85-4d4eeccf93ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:32.177220 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-935f6ed2-0980-4aa0-9a85-4d4eeccf93ce None None] MFA Rules not processed for user `9d372256c4844f7cb5affbebbf2571c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:32.178393 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-935f6ed2-0980-4aa0-9a85-4d4eeccf93ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:32.183717 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:32.249847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1408/2802] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:32.263750 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8498b8c-5f5f-48cb-8639-a8313b74193f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:32.267290 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d8498b8c-5f5f-48cb-8639-a8313b74193f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:32.339974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8498b8c-5f5f-48cb-8639-a8313b74193f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:32.346000 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049882c1-e229-4134-90a8-beb6769e407a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:32.346675 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049882c1-e229-4134-90a8-beb6769e407a service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:32.346961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049882c1-e229-4134-90a8-beb6769e407a service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:32.347238 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049882c1-e229-4134-90a8-beb6769e407a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:32.358367 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049882c1-e229-4134-90a8-beb6769e407a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:32.360200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049882c1-e229-4134-90a8-beb6769e407a service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:32.370492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1395/2803] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:32.774513 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f6ed2-0980-4aa0-9a85-4d4eeccf93ce None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:32.855867 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f6ed2-0980-4aa0-9a85-4d4eeccf93ce None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:32.861243 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:32.862144 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:32.862429 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:32.862688 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] PATH_INFO: `/v3/users/c3df9fab59bc4679a8ed5a808adb764f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:32.864006 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3df9fab59bc4679a8ed5a808adb764f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:32.866059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:32.919013 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3df9fab59bc4679a8ed5a808adb764f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:32.928837 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3df9fab59bc4679a8ed5a808adb764f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:32.936403 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3df9fab59bc4679a8ed5a808adb764f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:32.937030 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3df9fab59bc4679a8ed5a808adb764f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:32.948848 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3df9fab59bc4679a8ed5a808adb764f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:32.955515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1409/2804] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:32 2022] DELETE /identity/v3/users/c3df9fab59bc4679a8ed5a808adb764f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:32.960463 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da5882-653f-418c-85e0-3b5e30571af3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:32.960794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da5882-653f-418c-85e0-3b5e30571af3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:32.961224 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da5882-653f-418c-85e0-3b5e30571af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:32.981312 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-12da5882-653f-418c-85e0-3b5e30571af3 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:32.982210 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-12da5882-653f-418c-85e0-3b5e30571af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:33.047158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1396/2805] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:33.061398 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:33.063041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:33.094203 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:33.149040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3fa98-222b-459a-8478-6dd1ad6dc6fa admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:33.154074 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886a8fe-9d9e-47e4-ad77-7a0c9039c116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:33.154846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886a8fe-9d9e-47e4-ad77-7a0c9039c116 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:33.155158 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886a8fe-9d9e-47e4-ad77-7a0c9039c116 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:33.155473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886a8fe-9d9e-47e4-ad77-7a0c9039c116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:33.167085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886a8fe-9d9e-47e4-ad77-7a0c9039c116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:33.168281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886a8fe-9d9e-47e4-ad77-7a0c9039c116 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:33.179395 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1410/2806] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:33.485344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da5882-653f-418c-85e0-3b5e30571af3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:33.487070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-12da5882-653f-418c-85e0-3b5e30571af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:33.501599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da5882-653f-418c-85e0-3b5e30571af3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:33.507422 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:33.508344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:33.508636 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:33.508911 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] PATH_INFO: `/v3/projects/eb455367cec9457c949956902e6494a0` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:33.510255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb455367cec9457c949956902e6494a0)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:33.512241 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:33.564384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb455367cec9457c949956902e6494a0', 'request_id': 'req-27e438db-05ea-422b-b8ae-67b513ec1b06'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:33.570297 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb455367cec9457c949956902e6494a0', 'request_id': 'req-27e438db-05ea-422b-b8ae-67b513ec1b06'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:33.571420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:33.581055 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb455367cec9457c949956902e6494a0', 'request_id': 'req-27e438db-05ea-422b-b8ae-67b513ec1b06'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:33.601397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] The token cache is being invalidate because project eb455367cec9457c949956902e6494a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:33.601732 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:33.602397 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:33.603993 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1397/2807] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:33 2022] DELETE /identity/v3/projects/eb455367cec9457c949956902e6494a0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:33.608235 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886a8fe-9d9e-47e4-ad77-7a0c9039c116 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:33.619187 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886a8fe-9d9e-47e4-ad77-7a0c9039c116 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:33.689659 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed309e0-7c6e-4fe7-bcb2-77e8f4272a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:33.690856 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed309e0-7c6e-4fe7-bcb2-77e8f4272a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:33.691307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed309e0-7c6e-4fe7-bcb2-77e8f4272a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:33.691680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed309e0-7c6e-4fe7-bcb2-77e8f4272a41 admin admin] PATH_INFO: `/v3/users/9d372256c4844f7cb5affbebbf2571c3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:33.693029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed309e0-7c6e-4fe7-bcb2-77e8f4272a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d372256c4844f7cb5affbebbf2571c3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:33.695543 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed309e0-7c6e-4fe7-bcb2-77e8f4272a41 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:33.758609 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aed309e0-7c6e-4fe7-bcb2-77e8f4272a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d372256c4844f7cb5affbebbf2571c3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:33.773873 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aed309e0-7c6e-4fe7-bcb2-77e8f4272a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d372256c4844f7cb5affbebbf2571c3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:33.784152 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aed309e0-7c6e-4fe7-bcb2-77e8f4272a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d372256c4844f7cb5affbebbf2571c3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:33.784970 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aed309e0-7c6e-4fe7-bcb2-77e8f4272a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d372256c4844f7cb5affbebbf2571c3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:33.808888 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aed309e0-7c6e-4fe7-bcb2-77e8f4272a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d372256c4844f7cb5affbebbf2571c3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:33.819820 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1411/2808] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:33 2022] DELETE /identity/v3/users/9d372256c4844f7cb5affbebbf2571c3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:34.301341 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:34.303952 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:34.321143 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e438db-05ea-422b-b8ae-67b513ec1b06 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:34.429497 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d5f7c-d13f-48d3-b941-2f7b8847fee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:34.431002 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d5f7c-d13f-48d3-b941-2f7b8847fee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:34.431527 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d5f7c-d13f-48d3-b941-2f7b8847fee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:34.432015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d5f7c-d13f-48d3-b941-2f7b8847fee2 admin admin] PATH_INFO: `/v3/projects/afaaaf5951b84450b0fe4fac848ced8f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:34.433697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d5f7c-d13f-48d3-b941-2f7b8847fee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afaaaf5951b84450b0fe4fac848ced8f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:34.435956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d5f7c-d13f-48d3-b941-2f7b8847fee2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:34.540970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-635d5f7c-d13f-48d3-b941-2f7b8847fee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afaaaf5951b84450b0fe4fac848ced8f', 'request_id': 'req-635d5f7c-d13f-48d3-b941-2f7b8847fee2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:34.551086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-635d5f7c-d13f-48d3-b941-2f7b8847fee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afaaaf5951b84450b0fe4fac848ced8f', 'request_id': 'req-635d5f7c-d13f-48d3-b941-2f7b8847fee2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:34.565500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-635d5f7c-d13f-48d3-b941-2f7b8847fee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afaaaf5951b84450b0fe4fac848ced8f', 'request_id': 'req-635d5f7c-d13f-48d3-b941-2f7b8847fee2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:34.595577 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-635d5f7c-d13f-48d3-b941-2f7b8847fee2 admin admin] The token cache is being invalidate because project afaaaf5951b84450b0fe4fac848ced8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:34.596104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-635d5f7c-d13f-48d3-b941-2f7b8847fee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:34.596670 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-635d5f7c-d13f-48d3-b941-2f7b8847fee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:34.601671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1398/2809] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:34 2022] DELETE /identity/v3/projects/afaaaf5951b84450b0fe4fac848ced8f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:34.607548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b500a7a-e25b-439a-9b00-400d3d0a629d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:34.608040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b500a7a-e25b-439a-9b00-400d3d0a629d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:34.608364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b500a7a-e25b-439a-9b00-400d3d0a629d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:34.639872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5b500a7a-e25b-439a-9b00-400d3d0a629d None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:34.642815 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5b500a7a-e25b-439a-9b00-400d3d0a629d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:34.709484 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:34.758585 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1412/2810] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:34.767015 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c96509-b6ec-4f95-bc69-ee6cb26dbce0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:34.767469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c96509-b6ec-4f95-bc69-ee6cb26dbce0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:34.767609 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c96509-b6ec-4f95-bc69-ee6cb26dbce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:34.815969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d8c96509-b6ec-4f95-bc69-ee6cb26dbce0 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:34.818328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d8c96509-b6ec-4f95-bc69-ee6cb26dbce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:34.829075 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1399/2811] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:34.833033 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b500a7a-e25b-439a-9b00-400d3d0a629d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:34.848379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b500a7a-e25b-439a-9b00-400d3d0a629d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:34.859471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d563a2-9793-4eac-95cd-014308972f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:34.860467 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d563a2-9793-4eac-95cd-014308972f17 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:34.861823 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d563a2-9793-4eac-95cd-014308972f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:34.862307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d563a2-9793-4eac-95cd-014308972f17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:34.864768 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d563a2-9793-4eac-95cd-014308972f17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:12:34.865377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d563a2-9793-4eac-95cd-014308972f17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:34.868246 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d563a2-9793-4eac-95cd-014308972f17 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:34.889757 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1413/2812] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:12:34.891628 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c96509-b6ec-4f95-bc69-ee6cb26dbce0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:34.903804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c96509-b6ec-4f95-bc69-ee6cb26dbce0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:34.912052 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ba8bc-bcc8-4925-b1ad-bdea7fc38101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:34.913216 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ba8bc-bcc8-4925-b1ad-bdea7fc38101 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:34.913509 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ba8bc-bcc8-4925-b1ad-bdea7fc38101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:34.913896 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ba8bc-bcc8-4925-b1ad-bdea7fc38101 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:34.921308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ba8bc-bcc8-4925-b1ad-bdea7fc38101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:34.924365 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ba8bc-bcc8-4925-b1ad-bdea7fc38101 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:34.975466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1400/2813] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:34 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:34.977351 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d563a2-9793-4eac-95cd-014308972f17 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:34.991939 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d563a2-9793-4eac-95cd-014308972f17 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:35.138914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019e012-d81b-4910-9c80-adf93891ef46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:35.140105 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019e012-d81b-4910-9c80-adf93891ef46 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:35.140600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019e012-d81b-4910-9c80-adf93891ef46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:35.141091 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019e012-d81b-4910-9c80-adf93891ef46 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:35.142216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019e012-d81b-4910-9c80-adf93891ef46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:35.144699 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019e012-d81b-4910-9c80-adf93891ef46 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:35.201346 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1414/2814] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:34 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:35.205448 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ba8bc-bcc8-4925-b1ad-bdea7fc38101 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:35.220386 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ba8bc-bcc8-4925-b1ad-bdea7fc38101 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:35.227302 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d24b6-c3cb-45f7-858c-ec41b3bb1534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:35.229152 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d24b6-c3cb-45f7-858c-ec41b3bb1534 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:35.229484 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d24b6-c3cb-45f7-858c-ec41b3bb1534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:35.230158 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d24b6-c3cb-45f7-858c-ec41b3bb1534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:35.231280 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d24b6-c3cb-45f7-858c-ec41b3bb1534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:35.234852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d24b6-c3cb-45f7-858c-ec41b3bb1534 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:35.258064 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1401/2815] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:35.259041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019e012-d81b-4910-9c80-adf93891ef46 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:35.270629 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019e012-d81b-4910-9c80-adf93891ef46 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:35.278204 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fd2d7-8b70-47dd-86cb-82d607e7f74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:35.278979 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd2d7-8b70-47dd-86cb-82d607e7f74f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:35.279252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd2d7-8b70-47dd-86cb-82d607e7f74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:35.279548 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd2d7-8b70-47dd-86cb-82d607e7f74f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:35.280281 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fd2d7-8b70-47dd-86cb-82d607e7f74f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:35.283604 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fd2d7-8b70-47dd-86cb-82d607e7f74f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:35.305034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d24b6-c3cb-45f7-858c-ec41b3bb1534 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:35.313804 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1415/2816] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:35.327665 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d24b6-c3cb-45f7-858c-ec41b3bb1534 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:35.362145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0967a4-e8fe-4e31-9aa4-07b49d9a440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:35.368063 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0967a4-e8fe-4e31-9aa4-07b49d9a440f admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:35.368435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0967a4-e8fe-4e31-9aa4-07b49d9a440f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:35.372979 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0967a4-e8fe-4e31-9aa4-07b49d9a440f admin admin] PATH_INFO: `/v3/projects/437f1e878c9549aeb42dffd2ea8ebcaa/users/be053854380449a0958cc96025111689/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:35.406364 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0967a4-e8fe-4e31-9aa4-07b49d9a440f admin admin] RBAC: Authorizing `identity:create_grant(project_id=437f1e878c9549aeb42dffd2ea8ebcaa, user_id=be053854380449a0958cc96025111689, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:35.410363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0967a4-e8fe-4e31-9aa4-07b49d9a440f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:35.434091 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1402/2817] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:35 2022] PUT /identity/v3/projects/437f1e878c9549aeb42dffd2ea8ebcaa/users/be053854380449a0958cc96025111689/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:12:35.442818 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f21a3-9519-411b-acf8-cf2a0d68eed5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:35.443251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f21a3-9519-411b-acf8-cf2a0d68eed5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:35.443628 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f21a3-9519-411b-acf8-cf2a0d68eed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:35.464004 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0967a4-e8fe-4e31-9aa4-07b49d9a440f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:35.465918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cb0967a4-e8fe-4e31-9aa4-07b49d9a440f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:35.481689 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b59f21a3-9519-411b-acf8-cf2a0d68eed5 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:35.483252 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b59f21a3-9519-411b-acf8-cf2a0d68eed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:35.506330 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:35.566737 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1416/2818] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:35.577872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f21a3-9519-411b-acf8-cf2a0d68eed5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:35.579264 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0967a4-e8fe-4e31-9aa4-07b49d9a440f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:35.580081 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b59f21a3-9519-411b-acf8-cf2a0d68eed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:35.586460 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:35.588014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:35.588014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:35.588243 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] PATH_INFO: `/v3/users/d069346c79b74f7d9382893fd6046d19` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:35.589600 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d069346c79b74f7d9382893fd6046d19)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:35.591803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:35.664977 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd069346c79b74f7d9382893fd6046d19'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:35.666060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd069346c79b74f7d9382893fd6046d19'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:35.667542 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd069346c79b74f7d9382893fd6046d19'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:35.678322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd069346c79b74f7d9382893fd6046d19'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:35.678874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f21a3-9519-411b-acf8-cf2a0d68eed5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:35.683264 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d010434-4755-4c9d-b2aa-391a23239f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:35.684025 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d010434-4755-4c9d-b2aa-391a23239f2c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:35.684379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d010434-4755-4c9d-b2aa-391a23239f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:35.684925 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d010434-4755-4c9d-b2aa-391a23239f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:35.694437 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7d010434-4755-4c9d-b2aa-391a23239f2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:35.700132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:35.707936 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d010434-4755-4c9d-b2aa-391a23239f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:35.708469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd069346c79b74f7d9382893fd6046d19'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:35.710394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d010434-4755-4c9d-b2aa-391a23239f2c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:35.725258 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1417/2819] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:35.732471 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1403/2820] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:35 2022] DELETE /identity/v3/users/d069346c79b74f7d9382893fd6046d19 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:12:35.733910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3365c-1af2-4fd6-9391-221d7f6f1ad5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:35.734656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3365c-1af2-4fd6-9391-221d7f6f1ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:35.735321 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3365c-1af2-4fd6-9391-221d7f6f1ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:35.759591 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d5d3365c-1af2-4fd6-9391-221d7f6f1ad5 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:35.760902 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d5d3365c-1af2-4fd6-9391-221d7f6f1ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:35.888684 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1418/2821] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:35.901142 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:35.903965 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:36.011046 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5e73b-f6a1-4e09-8a00-93a2fa906332 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:36.017813 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061fb82-c504-40de-bcb9-19fab92a5415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:36.018863 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061fb82-c504-40de-bcb9-19fab92a5415 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:36.019798 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061fb82-c504-40de-bcb9-19fab92a5415 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:36.020805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061fb82-c504-40de-bcb9-19fab92a5415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:36.038706 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9061fb82-c504-40de-bcb9-19fab92a5415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:36.040724 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9061fb82-c504-40de-bcb9-19fab92a5415 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:36.063119 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1404/2822] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:36.585981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3365c-1af2-4fd6-9391-221d7f6f1ad5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:36.588472 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d5d3365c-1af2-4fd6-9391-221d7f6f1ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:36.635669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3365c-1af2-4fd6-9391-221d7f6f1ad5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:36.650188 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:36.652095 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:36.653852 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:36.654309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] PATH_INFO: `/v3/projects/cb70c085efb94ebd90b1f9d5add5d32b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:36.655879 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb70c085efb94ebd90b1f9d5add5d32b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:36.658814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:36.767816 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb70c085efb94ebd90b1f9d5add5d32b', 'request_id': 'req-c04003b7-19b9-41d0-a3e5-046233675f52'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:36.778751 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb70c085efb94ebd90b1f9d5add5d32b', 'request_id': 'req-c04003b7-19b9-41d0-a3e5-046233675f52'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:36.796372 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30c72b-969f-48bb-8a50-a0408dff1c18 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:36.797232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30c72b-969f-48bb-8a50-a0408dff1c18 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:36.797980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30c72b-969f-48bb-8a50-a0408dff1c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:36.817245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb70c085efb94ebd90b1f9d5add5d32b', 'request_id': 'req-c04003b7-19b9-41d0-a3e5-046233675f52'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:36.872913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] The token cache is being invalidate because project cb70c085efb94ebd90b1f9d5add5d32b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:36.878730 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:36.879491 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ad30c72b-969f-48bb-8a50-a0408dff1c18 None None] MFA Rules not processed for user `be053854380449a0958cc96025111689`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:36.880057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:36.882622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ad30c72b-969f-48bb-8a50-a0408dff1c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:36.886716 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1419/2823] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:36 2022] DELETE /identity/v3/projects/cb70c085efb94ebd90b1f9d5add5d32b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:36.961407 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:37.171694 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1405/2824] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:12:37.175132 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:37.177654 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:37.290790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04003b7-19b9-41d0-a3e5-046233675f52 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:37.296251 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2be6b3-4a7a-4344-9b32-169b9ab24662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:37.296887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2be6b3-4a7a-4344-9b32-169b9ab24662 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:37.297167 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2be6b3-4a7a-4344-9b32-169b9ab24662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:37.297440 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2be6b3-4a7a-4344-9b32-169b9ab24662 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:37.298179 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2be6b3-4a7a-4344-9b32-169b9ab24662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:37.300377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2be6b3-4a7a-4344-9b32-169b9ab24662 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:37.338245 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1420/2825] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:37 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:37.339409 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30c72b-969f-48bb-8a50-a0408dff1c18 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:37.359875 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30c72b-969f-48bb-8a50-a0408dff1c18 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:37.462126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6476a9-915c-4b67-9646-09debdccffaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:37.463770 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6476a9-915c-4b67-9646-09debdccffaa admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:37.464419 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6476a9-915c-4b67-9646-09debdccffaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:37.465202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6476a9-915c-4b67-9646-09debdccffaa admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:37.466625 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6476a9-915c-4b67-9646-09debdccffaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:37.470076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6476a9-915c-4b67-9646-09debdccffaa admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:37.505276 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1406/2826] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:37 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:12:37.506639 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2be6b3-4a7a-4344-9b32-169b9ab24662 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:37.513216 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:37.522580 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2be6b3-4a7a-4344-9b32-169b9ab24662 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:37.532774 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672392f6-2462-4113-bf02-2543fc1e437e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:37.534017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672392f6-2462-4113-bf02-2543fc1e437e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:37.534539 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672392f6-2462-4113-bf02-2543fc1e437e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:37.536329 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672392f6-2462-4113-bf02-2543fc1e437e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:37.536497 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672392f6-2462-4113-bf02-2543fc1e437e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:37.538387 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672392f6-2462-4113-bf02-2543fc1e437e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:37.551428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1421/2827] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:37.556424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6476a9-915c-4b67-9646-09debdccffaa admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:37.573466 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6476a9-915c-4b67-9646-09debdccffaa admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:37.578495 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e2f57-7cfe-4067-a960-cc7b9b41c845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:37.579420 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e2f57-7cfe-4067-a960-cc7b9b41c845 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:37.579697 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e2f57-7cfe-4067-a960-cc7b9b41c845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:37.579971 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e2f57-7cfe-4067-a960-cc7b9b41c845 admin admin] PATH_INFO: `/v3/projects/21ad9648f7e2437c9f47c4efbc9cd1f1/users/805173d03b5f4efc8e6b6db01e902809/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:37.601412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e2f57-7cfe-4067-a960-cc7b9b41c845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ad9648f7e2437c9f47c4efbc9cd1f1, user_id=805173d03b5f4efc8e6b6db01e902809, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:37.604598 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e2f57-7cfe-4067-a960-cc7b9b41c845 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:37.632212 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1407/2828] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:37 2022] PUT /identity/v3/projects/21ad9648f7e2437c9f47c4efbc9cd1f1/users/805173d03b5f4efc8e6b6db01e902809/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:37.636191 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672392f6-2462-4113-bf02-2543fc1e437e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:37.650057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672392f6-2462-4113-bf02-2543fc1e437e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:37.744755 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c1fff-61a7-42ca-9e75-f8447768d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:37.745594 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c1fff-61a7-42ca-9e75-f8447768d6df admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:37.745943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c1fff-61a7-42ca-9e75-f8447768d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:37.746333 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c1fff-61a7-42ca-9e75-f8447768d6df admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:37.747149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c1fff-61a7-42ca-9e75-f8447768d6df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:37.748882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c1fff-61a7-42ca-9e75-f8447768d6df admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:37.764261 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1422/2829] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:37.768593 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e2f57-7cfe-4067-a960-cc7b9b41c845 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:37.779532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e2f57-7cfe-4067-a960-cc7b9b41c845 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:37.787266 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e72b15-554b-4afa-a6de-48a50b727fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:37.788207 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e72b15-554b-4afa-a6de-48a50b727fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:37.788525 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e72b15-554b-4afa-a6de-48a50b727fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:37.788817 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e72b15-554b-4afa-a6de-48a50b727fd1 admin admin] PATH_INFO: `/v3/projects/21ad9648f7e2437c9f47c4efbc9cd1f1/users/805173d03b5f4efc8e6b6db01e902809/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:37.792739 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e72b15-554b-4afa-a6de-48a50b727fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ad9648f7e2437c9f47c4efbc9cd1f1, user_id=805173d03b5f4efc8e6b6db01e902809, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:37.795986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e72b15-554b-4afa-a6de-48a50b727fd1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:37.809943 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1408/2830] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:37 2022] PUT /identity/v3/projects/21ad9648f7e2437c9f47c4efbc9cd1f1/users/805173d03b5f4efc8e6b6db01e902809/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:38.226764 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a7b09-d034-40b5-b55c-8ae31d82b8f9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:38.227385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a7b09-d034-40b5-b55c-8ae31d82b8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:38.227385 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a7b09-d034-40b5-b55c-8ae31d82b8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:38.253276 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b20a7b09-d034-40b5-b55c-8ae31d82b8f9 None None] MFA Rules not processed for user `fa28f0e0b3b0488faf7db7bf7dc3854b`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:38.254827 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b20a7b09-d034-40b5-b55c-8ae31d82b8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:38.380184 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1423/2831] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:38.798182 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9719150-33e3-46ee-b6db-59ff36aaec4f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:38.798552 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9719150-33e3-46ee-b6db-59ff36aaec4f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:38.798744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9719150-33e3-46ee-b6db-59ff36aaec4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:38.853187 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d9719150-33e3-46ee-b6db-59ff36aaec4f None None] MFA Rules not processed for user `805173d03b5f4efc8e6b6db01e902809`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:38.857202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d9719150-33e3-46ee-b6db-59ff36aaec4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:39.022062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a7b09-d034-40b5-b55c-8ae31d82b8f9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:39.024681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b20a7b09-d034-40b5-b55c-8ae31d82b8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:39.037931 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1409/2832] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:39.110531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a7b09-d034-40b5-b55c-8ae31d82b8f9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:39.117185 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1964e46-c5dc-4cfe-b51c-de8d0537952d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:39.117984 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1964e46-c5dc-4cfe-b51c-de8d0537952d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:39.118271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1964e46-c5dc-4cfe-b51c-de8d0537952d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:39.118549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1964e46-c5dc-4cfe-b51c-de8d0537952d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:39.200356 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:39.236072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1964e46-c5dc-4cfe-b51c-de8d0537952d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:39.237592 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1964e46-c5dc-4cfe-b51c-de8d0537952d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:39.254313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1424/2833] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:12:40.825590 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9719150-33e3-46ee-b6db-59ff36aaec4f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:40.830801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d9719150-33e3-46ee-b6db-59ff36aaec4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:40.842337 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:40.852010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9719150-33e3-46ee-b6db-59ff36aaec4f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:40.857970 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb75a2-5797-4a34-8f35-3c17b9703fee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061ffcf242ff4155b92daa8c3888fd10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:40.858761 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb75a2-5797-4a34-8f35-3c17b9703fee service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:40.859064 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb75a2-5797-4a34-8f35-3c17b9703fee service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:40.859315 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb75a2-5797-4a34-8f35-3c17b9703fee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:40.871511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb75a2-5797-4a34-8f35-3c17b9703fee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:40.873700 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb75a2-5797-4a34-8f35-3c17b9703fee service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:40.898018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1410/2834] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:44.519728 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fcbbd-2423-4190-a0f0-d5e1506b629e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:44.520180 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fcbbd-2423-4190-a0f0-d5e1506b629e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:44.520656 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fcbbd-2423-4190-a0f0-d5e1506b629e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:44.541054 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-de2fcbbd-2423-4190-a0f0-d5e1506b629e None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:44.542319 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-de2fcbbd-2423-4190-a0f0-d5e1506b629e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:44.604493 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1425/2835] 10.222.0.43 () {58 vars in 967 bytes} [Mon Jul 4 08:12:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:44.618183 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb75a2-5797-4a34-8f35-3c17b9703fee service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:44.620306 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e8bb75a2-5797-4a34-8f35-3c17b9703fee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:44.701408 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb75a2-5797-4a34-8f35-3c17b9703fee service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:44.706802 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc90e6-ab83-4311-92a4-11f215bd0a71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:44.707467 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc90e6-ab83-4311-92a4-11f215bd0a71 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:44.707778 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc90e6-ab83-4311-92a4-11f215bd0a71 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:44.708051 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc90e6-ab83-4311-92a4-11f215bd0a71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:44.718322 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc90e6-ab83-4311-92a4-11f215bd0a71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:44.719384 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc90e6-ab83-4311-92a4-11f215bd0a71 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:44.730423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1411/2836] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:12:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 08:12:46.849675 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fcbbd-2423-4190-a0f0-d5e1506b629e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:46.851819 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-de2fcbbd-2423-4190-a0f0-d5e1506b629e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:46.948793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fcbbd-2423-4190-a0f0-d5e1506b629e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:46.954051 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b560e-5af0-49b1-a439-3515d0ec48e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:46.955029 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b560e-5af0-49b1-a439-3515d0ec48e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:46.955275 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b560e-5af0-49b1-a439-3515d0ec48e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:46.955551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b560e-5af0-49b1-a439-3515d0ec48e5 admin admin] PATH_INFO: `/v3/users/0224639fe5d24820bdead1bba468ddda` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:46.957151 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b560e-5af0-49b1-a439-3515d0ec48e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0224639fe5d24820bdead1bba468ddda)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:46.959305 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b560e-5af0-49b1-a439-3515d0ec48e5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:47.022100 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2f0b560e-5af0-49b1-a439-3515d0ec48e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0224639fe5d24820bdead1bba468ddda'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:47.034155 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2f0b560e-5af0-49b1-a439-3515d0ec48e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0224639fe5d24820bdead1bba468ddda'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:47.045526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2f0b560e-5af0-49b1-a439-3515d0ec48e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0224639fe5d24820bdead1bba468ddda'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:47.047364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2f0b560e-5af0-49b1-a439-3515d0ec48e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0224639fe5d24820bdead1bba468ddda'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:47.064123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2f0b560e-5af0-49b1-a439-3515d0ec48e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0224639fe5d24820bdead1bba468ddda'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:47.073242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1426/2837] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:46 2022] DELETE /identity/v3/users/0224639fe5d24820bdead1bba468ddda => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:47.294859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc90e6-ab83-4311-92a4-11f215bd0a71 service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:47.308365 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc90e6-ab83-4311-92a4-11f215bd0a71 service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:47.391285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:47.392256 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:47.392622 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:47.392799 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c admin admin] PATH_INFO: `/v3/projects/bfb2bf1afead415b9d6ee79ad836197d` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:47.394363 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfb2bf1afead415b9d6ee79ad836197d)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:47.396250 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:47.455849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfb2bf1afead415b9d6ee79ad836197d', 'request_id': 'req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:47.462307 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfb2bf1afead415b9d6ee79ad836197d', 'request_id': 'req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:47.470294 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfb2bf1afead415b9d6ee79ad836197d', 'request_id': 'req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:47.486887 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c admin admin] The token cache is being invalidate because project bfb2bf1afead415b9d6ee79ad836197d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:47.487252 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:47.487950 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8fcb7277-4df6-41f0-ab63-ea7590a3e00c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:47.489614 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1412/2838] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:47 2022] DELETE /identity/v3/projects/bfb2bf1afead415b9d6ee79ad836197d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:47.495124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8aa78-3a2e-48b1-a084-e7e961b41997 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:47.495422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8aa78-3a2e-48b1-a084-e7e961b41997 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:47.495703 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8aa78-3a2e-48b1-a084-e7e961b41997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:47.515742 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d4d8aa78-3a2e-48b1-a084-e7e961b41997 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:47.517041 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d4d8aa78-3a2e-48b1-a084-e7e961b41997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:47.523531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:47.609349 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1427/2839] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:47.703508 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8f153-729b-45c0-b387-a7afb79fc3c7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:47.703753 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8f153-729b-45c0-b387-a7afb79fc3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:47.703957 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8f153-729b-45c0-b387-a7afb79fc3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:47.730382 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1ec8f153-729b-45c0-b387-a7afb79fc3c7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:47.732328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1ec8f153-729b-45c0-b387-a7afb79fc3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:47.746066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1413/2840] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:47.750987 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8aa78-3a2e-48b1-a084-e7e961b41997 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:47.761985 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8aa78-3a2e-48b1-a084-e7e961b41997 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:47.767471 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d70e40-5b08-4d32-918b-cd5d9893a74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:47.768168 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d70e40-5b08-4d32-918b-cd5d9893a74b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:47.768449 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d70e40-5b08-4d32-918b-cd5d9893a74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:47.768725 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d70e40-5b08-4d32-918b-cd5d9893a74b admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:47.769463 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d70e40-5b08-4d32-918b-cd5d9893a74b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:12:47.769806 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d70e40-5b08-4d32-918b-cd5d9893a74b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:47.771442 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d70e40-5b08-4d32-918b-cd5d9893a74b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:47.786906 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1428/2841] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:12:47.796979 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8f153-729b-45c0-b387-a7afb79fc3c7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:47.804076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:47.814956 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8f153-729b-45c0-b387-a7afb79fc3c7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:47.821010 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc68c9-0e15-4035-8208-2a8276cf9533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:47.821715 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc68c9-0e15-4035-8208-2a8276cf9533 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:47.821999 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc68c9-0e15-4035-8208-2a8276cf9533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:47.822275 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc68c9-0e15-4035-8208-2a8276cf9533 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:47.823003 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc68c9-0e15-4035-8208-2a8276cf9533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:47.824976 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc68c9-0e15-4035-8208-2a8276cf9533 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:47.853377 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1414/2842] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:12:47 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:47.858040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d70e40-5b08-4d32-918b-cd5d9893a74b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:47.875206 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d70e40-5b08-4d32-918b-cd5d9893a74b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:47.953000 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107a634-ff23-45c0-ae23-0672dfbdaced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:47.953808 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107a634-ff23-45c0-ae23-0672dfbdaced admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:47.954037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107a634-ff23-45c0-ae23-0672dfbdaced admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:47.954320 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107a634-ff23-45c0-ae23-0672dfbdaced admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:47.955040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107a634-ff23-45c0-ae23-0672dfbdaced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:47.957034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107a634-ff23-45c0-ae23-0672dfbdaced admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:47.984307 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1429/2843] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:12:47 2022] POST /identity/v3/users => generated 285 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:12:47.988923 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc68c9-0e15-4035-8208-2a8276cf9533 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:48.000782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc68c9-0e15-4035-8208-2a8276cf9533 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:48.006864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3636b69-6e36-4803-885b-ebaa7abb7b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:48.007563 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3636b69-6e36-4803-885b-ebaa7abb7b06 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:48.007852 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3636b69-6e36-4803-885b-ebaa7abb7b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:48.008106 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3636b69-6e36-4803-885b-ebaa7abb7b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:48.008859 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3636b69-6e36-4803-885b-ebaa7abb7b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:48.010559 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3636b69-6e36-4803-885b-ebaa7abb7b06 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:48.024620 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1415/2844] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:48.029457 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107a634-ff23-45c0-ae23-0672dfbdaced admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:48.040880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107a634-ff23-45c0-ae23-0672dfbdaced admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:48.045942 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501db24a-4170-4973-a176-f6698475e06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:48.046633 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db24a-4170-4973-a176-f6698475e06c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:48.047017 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db24a-4170-4973-a176-f6698475e06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:48.047298 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db24a-4170-4973-a176-f6698475e06c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:48.048042 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501db24a-4170-4973-a176-f6698475e06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:48.049891 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501db24a-4170-4973-a176-f6698475e06c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:48.063599 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1430/2845] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:12:48.068328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3636b69-6e36-4803-885b-ebaa7abb7b06 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:48.079808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3636b69-6e36-4803-885b-ebaa7abb7b06 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:48.084727 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac1d29-22e5-4e9c-a5ed-23ca0834ddfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:48.085658 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac1d29-22e5-4e9c-a5ed-23ca0834ddfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:48.085947 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac1d29-22e5-4e9c-a5ed-23ca0834ddfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:48.086225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac1d29-22e5-4e9c-a5ed-23ca0834ddfd admin admin] PATH_INFO: `/v3/projects/6c167fcc4ddc43d1afc5b8cf82e06c0f/users/3d2dd3038db6470490505cd4e078d14c/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:48.103525 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac1d29-22e5-4e9c-a5ed-23ca0834ddfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c167fcc4ddc43d1afc5b8cf82e06c0f, user_id=3d2dd3038db6470490505cd4e078d14c, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:48.106260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac1d29-22e5-4e9c-a5ed-23ca0834ddfd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:48.124088 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1416/2846] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:12:48 2022] PUT /identity/v3/projects/6c167fcc4ddc43d1afc5b8cf82e06c0f/users/3d2dd3038db6470490505cd4e078d14c/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:48.135410 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587d20e-d55b-4fe8-bdf9-bf629dd99fa6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:48.135704 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587d20e-d55b-4fe8-bdf9-bf629dd99fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:48.135988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587d20e-d55b-4fe8-bdf9-bf629dd99fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:48.175956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2587d20e-d55b-4fe8-bdf9-bf629dd99fa6 None None] MFA Rules not processed for user `3d2dd3038db6470490505cd4e078d14c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:12:48.177946 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2587d20e-d55b-4fe8-bdf9-bf629dd99fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:48.266525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1431/2847] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:12:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:12:48.279939 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac1d29-22e5-4e9c-a5ed-23ca0834ddfd admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:48.282808 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-58ac1d29-22e5-4e9c-a5ed-23ca0834ddfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:48.364232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac1d29-22e5-4e9c-a5ed-23ca0834ddfd admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:48.368729 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67b3bc-d92b-4799-86f4-1dab92b24bfc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a122422a45446c9a63d67b4f931fa47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:48.369393 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67b3bc-d92b-4799-86f4-1dab92b24bfc service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:48.369717 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67b3bc-d92b-4799-86f4-1dab92b24bfc service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:48.370031 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67b3bc-d92b-4799-86f4-1dab92b24bfc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:48.381499 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67b3bc-d92b-4799-86f4-1dab92b24bfc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:48.383134 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67b3bc-d92b-4799-86f4-1dab92b24bfc service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:48.394605 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1417/2848] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:48 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:12:55.195044 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587d20e-d55b-4fe8-bdf9-bf629dd99fa6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:55.197344 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2587d20e-d55b-4fe8-bdf9-bf629dd99fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:55.212111 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:55.303528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587d20e-d55b-4fe8-bdf9-bf629dd99fa6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:55.309253 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:55.310227 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:55.310538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:55.310810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] PATH_INFO: `/v3/users/be053854380449a0958cc96025111689` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:55.312138 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be053854380449a0958cc96025111689)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:55.314139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:55.376211 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be053854380449a0958cc96025111689'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.392767 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be053854380449a0958cc96025111689'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.401923 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be053854380449a0958cc96025111689'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.402759 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be053854380449a0958cc96025111689'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.416024 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be053854380449a0958cc96025111689'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.422851 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1432/2849] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:55 2022] DELETE /identity/v3/users/be053854380449a0958cc96025111689 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:55.614891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67b3bc-d92b-4799-86f4-1dab92b24bfc service glance] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:55.626561 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67b3bc-d92b-4799-86f4-1dab92b24bfc service glance] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:55.687795 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:55.688580 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:55.688862 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:55.689137 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] PATH_INFO: `/v3/projects/437f1e878c9549aeb42dffd2ea8ebcaa` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:55.690366 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=437f1e878c9549aeb42dffd2ea8ebcaa)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:55.692359 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:55.751844 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437f1e878c9549aeb42dffd2ea8ebcaa', 'request_id': 'req-6952f382-ac6f-4b26-9818-ac5900e5c491'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.753469 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:55.760278 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437f1e878c9549aeb42dffd2ea8ebcaa', 'request_id': 'req-6952f382-ac6f-4b26-9818-ac5900e5c491'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.767699 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437f1e878c9549aeb42dffd2ea8ebcaa', 'request_id': 'req-6952f382-ac6f-4b26-9818-ac5900e5c491'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.786145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] The token cache is being invalidate because project 437f1e878c9549aeb42dffd2ea8ebcaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:55.786448 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.787254 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.788987 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1418/2850] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:55 2022] DELETE /identity/v3/projects/437f1e878c9549aeb42dffd2ea8ebcaa => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:55.793404 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:55.795008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:55.808786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d289008-dae6-45eb-8190-767843397660 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:55.878829 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:55.879980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:55.880286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:55.880622 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] PATH_INFO: `/v3/users/805173d03b5f4efc8e6b6db01e902809` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:55.882276 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=805173d03b5f4efc8e6b6db01e902809)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:55.884520 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:55.919354 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:12:55.942365 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805173d03b5f4efc8e6b6db01e902809'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.954313 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805173d03b5f4efc8e6b6db01e902809'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.963270 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805173d03b5f4efc8e6b6db01e902809'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.963956 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805173d03b5f4efc8e6b6db01e902809'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.976684 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805173d03b5f4efc8e6b6db01e902809'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:55.982832 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1433/2851] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:12:55 2022] DELETE /identity/v3/users/805173d03b5f4efc8e6b6db01e902809 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:56.150406 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:56.151854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:56.188107 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:12:56.189635 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:56.228265 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952f382-ac6f-4b26-9818-ac5900e5c491 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:56.271584 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069d260-156d-4217-88e6-d458b1d88ab3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:12:56.277189 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:56.278123 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:56.278401 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:56.278674 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] PATH_INFO: `/v3/projects/21ad9648f7e2437c9f47c4efbc9cd1f1` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:56.279903 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21ad9648f7e2437c9f47c4efbc9cd1f1)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:56.281844 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:56.291979 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23755eb1-a289-4a20-a57e-f4bbf15a7007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:12:56.292642 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23755eb1-a289-4a20-a57e-f4bbf15a7007 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:12:56.292921 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23755eb1-a289-4a20-a57e-f4bbf15a7007 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:12:56.293192 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23755eb1-a289-4a20-a57e-f4bbf15a7007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:12:56.295232 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-23755eb1-a289-4a20-a57e-f4bbf15a7007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:12:56.362920 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21ad9648f7e2437c9f47c4efbc9cd1f1', 'request_id': 'req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:56.369057 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21ad9648f7e2437c9f47c4efbc9cd1f1', 'request_id': 'req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:56.383694 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21ad9648f7e2437c9f47c4efbc9cd1f1', 'request_id': 'req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:56.385412 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23755eb1-a289-4a20-a57e-f4bbf15a7007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:12:56.387744 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23755eb1-a289-4a20-a57e-f4bbf15a7007 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:12:56.405023 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] The token cache is being invalidate because project 21ad9648f7e2437c9f47c4efbc9cd1f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:12:56.405422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:56.406364 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:12:56.408085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1434/2853] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:12:56 2022] DELETE /identity/v3/projects/21ad9648f7e2437c9f47c4efbc9cd1f1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:12:56.414963 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1419/2853] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:12:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:13:06.394682 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=70,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:13:06.395441 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=3,keystone:SELECT=76,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:13:21.279862 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:21.282271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:21.355436 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f199e0-aff6-4c6d-aa52-57ceb64a8693 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:21.360670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:21.361567 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:21.361872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:21.362143 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] PATH_INFO: `/v3/users/3d2dd3038db6470490505cd4e078d14c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:21.363357 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d2dd3038db6470490505cd4e078d14c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:21.365332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:21.423741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d2dd3038db6470490505cd4e078d14c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:13:21.435922 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d2dd3038db6470490505cd4e078d14c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:13:21.444128 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d2dd3038db6470490505cd4e078d14c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:13:21.444999 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d2dd3038db6470490505cd4e078d14c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:13:21.459136 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d2dd3038db6470490505cd4e078d14c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:13:21.466139 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1435/2854] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:13:21 2022] DELETE /identity/v3/users/3d2dd3038db6470490505cd4e078d14c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:13:21.471811 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeef3d3-be1b-44b4-8b22-61a8bac4ce0a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:21.472357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeef3d3-be1b-44b4-8b22-61a8bac4ce0a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:21.472357 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeef3d3-be1b-44b4-8b22-61a8bac4ce0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:21.491849 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-daeef3d3-be1b-44b4-8b22-61a8bac4ce0a None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:13:21.493096 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-daeef3d3-be1b-44b4-8b22-61a8bac4ce0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:21.561938 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1420/2855] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:13:21.572566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:21.574084 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:21.637371 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c993c-a926-45e5-a8c9-6844e6d5b6cd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:21.642526 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68af32-5751-4e9e-9218-79dee24e6412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:21.643210 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68af32-5751-4e9e-9218-79dee24e6412 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:21.643499 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68af32-5751-4e9e-9218-79dee24e6412 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:21.643774 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68af32-5751-4e9e-9218-79dee24e6412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:21.654921 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68af32-5751-4e9e-9218-79dee24e6412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:21.656550 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68af32-5751-4e9e-9218-79dee24e6412 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:21.668531 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1436/2856] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:13:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:13:21.966163 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeef3d3-be1b-44b4-8b22-61a8bac4ce0a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:21.967856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-daeef3d3-be1b-44b4-8b22-61a8bac4ce0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:21.981070 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeef3d3-be1b-44b4-8b22-61a8bac4ce0a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:21.989447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:21.989447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:21.989447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:21.989447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30 admin admin] PATH_INFO: `/v3/projects/6c167fcc4ddc43d1afc5b8cf82e06c0f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:21.990111 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c167fcc4ddc43d1afc5b8cf82e06c0f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:21.992225 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:22.057447 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c167fcc4ddc43d1afc5b8cf82e06c0f', 'request_id': 'req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:13:22.063632 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c167fcc4ddc43d1afc5b8cf82e06c0f', 'request_id': 'req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:13:22.070798 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c167fcc4ddc43d1afc5b8cf82e06c0f', 'request_id': 'req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:13:22.091305 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30 admin admin] The token cache is being invalidate because project 6c167fcc4ddc43d1afc5b8cf82e06c0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:13:22.091749 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:13:22.092691 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8ae8a86c-6f87-45f6-8817-a15ba87ffd30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:13:22.094346 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1421/2857] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:13:21 2022] DELETE /identity/v3/projects/6c167fcc4ddc43d1afc5b8cf82e06c0f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:13:22.099553 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8943e-a8ac-4874-8c10-9744f45cee59 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:22.099845 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8943e-a8ac-4874-8c10-9744f45cee59 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:22.100124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8943e-a8ac-4874-8c10-9744f45cee59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:22.118992 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1ec8943e-a8ac-4874-8c10-9744f45cee59 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:13:22.120217 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1ec8943e-a8ac-4874-8c10-9744f45cee59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:22.140223 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:13:22.188616 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1437/2858] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:13:22.198427 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf8e9a-087e-4b7a-af99-3efdeefa243b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:22.198667 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf8e9a-087e-4b7a-af99-3efdeefa243b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:22.199066 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf8e9a-087e-4b7a-af99-3efdeefa243b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:22.220755 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3fdf8e9a-087e-4b7a-af99-3efdeefa243b None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:13:22.222881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3fdf8e9a-087e-4b7a-af99-3efdeefa243b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:22.238117 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1422/2859] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:13:22.245118 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8943e-a8ac-4874-8c10-9744f45cee59 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:22.262094 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8943e-a8ac-4874-8c10-9744f45cee59 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:22.268910 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee4b10-d6fd-442d-bb07-6d2bec3805db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:22.269951 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee4b10-d6fd-442d-bb07-6d2bec3805db admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:22.270239 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee4b10-d6fd-442d-bb07-6d2bec3805db admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:22.270516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee4b10-d6fd-442d-bb07-6d2bec3805db admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:22.271299 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee4b10-d6fd-442d-bb07-6d2bec3805db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 08:13:22.271597 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee4b10-d6fd-442d-bb07-6d2bec3805db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:22.273294 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee4b10-d6fd-442d-bb07-6d2bec3805db admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:22.289914 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1438/2860] 10.222.0.43 () {58 vars in 1152 bytes} [Mon Jul 4 08:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 08:13:22.294709 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf8e9a-087e-4b7a-af99-3efdeefa243b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:22.305013 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf8e9a-087e-4b7a-af99-3efdeefa243b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:22.310024 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d6426-6400-4615-b906-beb6e22542ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:22.310815 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d6426-6400-4615-b906-beb6e22542ae admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:22.311100 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d6426-6400-4615-b906-beb6e22542ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:22.311375 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d6426-6400-4615-b906-beb6e22542ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:22.312093 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d6426-6400-4615-b906-beb6e22542ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:22.314079 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d6426-6400-4615-b906-beb6e22542ae admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:22.350086 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1423/2861] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:13:22 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:13:22.354486 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee4b10-d6fd-442d-bb07-6d2bec3805db admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:22.366680 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee4b10-d6fd-442d-bb07-6d2bec3805db admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:22.431690 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5166cb-92fc-4f1b-b10a-797ba62f7800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:22.432378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5166cb-92fc-4f1b-b10a-797ba62f7800 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:22.432664 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5166cb-92fc-4f1b-b10a-797ba62f7800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:22.432943 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5166cb-92fc-4f1b-b10a-797ba62f7800 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:22.433681 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5166cb-92fc-4f1b-b10a-797ba62f7800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:22.436059 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5166cb-92fc-4f1b-b10a-797ba62f7800 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:22.460255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1439/2862] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:13:22 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 08:13:22.464772 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d6426-6400-4615-b906-beb6e22542ae admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:22.476496 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d6426-6400-4615-b906-beb6e22542ae admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:22.483042 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f5bc0-5c93-43d8-8848-c822e5a4f268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:22.483919 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5bc0-5c93-43d8-8848-c822e5a4f268 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:22.484233 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5bc0-5c93-43d8-8848-c822e5a4f268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:22.484411 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5bc0-5c93-43d8-8848-c822e5a4f268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:22.485203 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f5bc0-5c93-43d8-8848-c822e5a4f268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:22.486868 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f5bc0-5c93-43d8-8848-c822e5a4f268 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:22.498652 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1424/2863] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:13:22.503528 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5166cb-92fc-4f1b-b10a-797ba62f7800 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:22.516417 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5166cb-92fc-4f1b-b10a-797ba62f7800 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:22.521516 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf28ea-0c86-446b-9047-c18e7aab4d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:22.522262 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf28ea-0c86-446b-9047-c18e7aab4d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:22.522551 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf28ea-0c86-446b-9047-c18e7aab4d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:22.522822 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf28ea-0c86-446b-9047-c18e7aab4d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:22.523562 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf28ea-0c86-446b-9047-c18e7aab4d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:22.525201 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf28ea-0c86-446b-9047-c18e7aab4d0f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:22.536225 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1440/2864] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:13:22.540803 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f5bc0-5c93-43d8-8848-c822e5a4f268 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:22.551440 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f5bc0-5c93-43d8-8848-c822e5a4f268 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:22.556221 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fa0c6-d98c-49da-a2df-1c33642f02a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:22.557112 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fa0c6-d98c-49da-a2df-1c33642f02a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:22.557389 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fa0c6-d98c-49da-a2df-1c33642f02a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:22.557687 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fa0c6-d98c-49da-a2df-1c33642f02a0 admin admin] PATH_INFO: `/v3/projects/fea21868e80c43aa81d1a9d277da4e9d/users/c11b4c72af804f68ad2bc19e91f11129/roles/3edce31396af478ea1ea7b76d33da530` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:22.573861 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fa0c6-d98c-49da-a2df-1c33642f02a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea21868e80c43aa81d1a9d277da4e9d, user_id=c11b4c72af804f68ad2bc19e91f11129, role_id=3edce31396af478ea1ea7b76d33da530)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:22.576568 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fa0c6-d98c-49da-a2df-1c33642f02a0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:22.591912 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1425/2865] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:13:22 2022] PUT /identity/v3/projects/fea21868e80c43aa81d1a9d277da4e9d/users/c11b4c72af804f68ad2bc19e91f11129/roles/3edce31396af478ea1ea7b76d33da530 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:13:22.597296 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce645d1-f6e9-4c5e-ac3c-3ba9b78391a7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:22.597645 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce645d1-f6e9-4c5e-ac3c-3ba9b78391a7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:22.597926 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce645d1-f6e9-4c5e-ac3c-3ba9b78391a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:22.618034 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7ce645d1-f6e9-4c5e-ac3c-3ba9b78391a7 None None] MFA Rules not processed for user `aec7b27e85c74490a82ad099b2e0743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:13:22.619403 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7ce645d1-f6e9-4c5e-ac3c-3ba9b78391a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:22.686669 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1441/2866] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:13:22.696905 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fa0c6-d98c-49da-a2df-1c33642f02a0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:22.698856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-148fa0c6-d98c-49da-a2df-1c33642f02a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:22.730281 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:13:22.770762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fa0c6-d98c-49da-a2df-1c33642f02a0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:22.774570 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0bf6e-09b8-4ab2-b618-e0a5bfb433e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:22.775223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0bf6e-09b8-4ab2-b618-e0a5bfb433e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:22.775511 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0bf6e-09b8-4ab2-b618-e0a5bfb433e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:22.775790 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0bf6e-09b8-4ab2-b618-e0a5bfb433e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:22.785897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0bf6e-09b8-4ab2-b618-e0a5bfb433e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:22.787355 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0bf6e-09b8-4ab2-b618-e0a5bfb433e1 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:22.798104 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1426/2867] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:13:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:13:27.226830 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff416ce-2ced-4235-8757-11e3b1d34ec8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:27.227544 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff416ce-2ced-4235-8757-11e3b1d34ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:27.227786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff416ce-2ced-4235-8757-11e3b1d34ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:27.263002 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9ff416ce-2ced-4235-8757-11e3b1d34ec8 None None] MFA Rules not processed for user `c11b4c72af804f68ad2bc19e91f11129`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:13:27.264644 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9ff416ce-2ced-4235-8757-11e3b1d34ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:27.286576 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:13:27.352705 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1442/2868] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:13:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:13:27.359837 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0bf6e-09b8-4ab2-b618-e0a5bfb433e1 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:27.373291 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0bf6e-09b8-4ab2-b618-e0a5bfb433e1 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:27.381258 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d8447-61a7-497a-9287-7555ce3974ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:27.381986 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d8447-61a7-497a-9287-7555ce3974ce admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:27.382621 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d8447-61a7-497a-9287-7555ce3974ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:27.382909 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d8447-61a7-497a-9287-7555ce3974ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:27.383630 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d8447-61a7-497a-9287-7555ce3974ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:27.385689 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d8447-61a7-497a-9287-7555ce3974ce admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:27.430502 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1427/2869] 10.222.0.43 () {60 vars in 1152 bytes} [Mon Jul 4 08:13:27 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:13:27.434872 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff416ce-2ced-4235-8757-11e3b1d34ec8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:27.448323 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff416ce-2ced-4235-8757-11e3b1d34ec8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:27.524293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128469fa-6d79-4584-a987-98973333753b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:27.525008 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128469fa-6d79-4584-a987-98973333753b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:27.525413 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128469fa-6d79-4584-a987-98973333753b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:27.525763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128469fa-6d79-4584-a987-98973333753b admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:27.526475 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128469fa-6d79-4584-a987-98973333753b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:27.529146 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128469fa-6d79-4584-a987-98973333753b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:27.557289 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1443/2870] 10.222.0.43 () {60 vars in 1143 bytes} [Mon Jul 4 08:13:27 2022] POST /identity/v3/users => generated 299 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 08:13:27.562014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d8447-61a7-497a-9287-7555ce3974ce admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:27.574231 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d8447-61a7-497a-9287-7555ce3974ce admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:27.581255 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5ea86-6355-4b0e-82e0-9f51146596ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:27.581974 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5ea86-6355-4b0e-82e0-9f51146596ae admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:27.582260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5ea86-6355-4b0e-82e0-9f51146596ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:27.582539 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5ea86-6355-4b0e-82e0-9f51146596ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:27.583272 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5ea86-6355-4b0e-82e0-9f51146596ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:27.584897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5ea86-6355-4b0e-82e0-9f51146596ae admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:27.596898 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1428/2871] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:13:27.601293 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128469fa-6d79-4584-a987-98973333753b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:27.613113 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128469fa-6d79-4584-a987-98973333753b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:27.618882 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab570b-6477-47d8-8352-f16ab237f918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:27.619814 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab570b-6477-47d8-8352-f16ab237f918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:27.620099 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab570b-6477-47d8-8352-f16ab237f918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:27.620379 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab570b-6477-47d8-8352-f16ab237f918 admin admin] PATH_INFO: `/v3/projects/33f007cbc1814baea8b60d8830cea9c4/users/0445562b148a44779be81a2eeece6097/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:27.643540 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab570b-6477-47d8-8352-f16ab237f918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33f007cbc1814baea8b60d8830cea9c4, user_id=0445562b148a44779be81a2eeece6097, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:27.646336 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab570b-6477-47d8-8352-f16ab237f918 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:27.665288 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1444/2872] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:13:27 2022] PUT /identity/v3/projects/33f007cbc1814baea8b60d8830cea9c4/users/0445562b148a44779be81a2eeece6097/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:13:27.669671 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5ea86-6355-4b0e-82e0-9f51146596ae admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:27.680838 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5ea86-6355-4b0e-82e0-9f51146596ae admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:27.742794 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dabc3b-a6a2-49bc-87fd-3c9c8b29560e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:27.743475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dabc3b-a6a2-49bc-87fd-3c9c8b29560e admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:27.743762 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dabc3b-a6a2-49bc-87fd-3c9c8b29560e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:27.744021 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dabc3b-a6a2-49bc-87fd-3c9c8b29560e admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:27.744754 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dabc3b-a6a2-49bc-87fd-3c9c8b29560e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:27.746422 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dabc3b-a6a2-49bc-87fd-3c9c8b29560e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:27.760219 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1429/2873] 10.222.0.43 () {58 vars in 1121 bytes} [Mon Jul 4 08:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 08:13:27.764600 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab570b-6477-47d8-8352-f16ab237f918 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:27.777395 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab570b-6477-47d8-8352-f16ab237f918 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:27.786720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79b3c3-7630-44cf-9ec0-62d827eb7709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:27.788019 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79b3c3-7630-44cf-9ec0-62d827eb7709 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:27.788411 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79b3c3-7630-44cf-9ec0-62d827eb7709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:27.788874 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79b3c3-7630-44cf-9ec0-62d827eb7709 admin admin] PATH_INFO: `/v3/projects/33f007cbc1814baea8b60d8830cea9c4/users/0445562b148a44779be81a2eeece6097/roles/e60a0103aa774cff9ee9b873bca64fb6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:27.792763 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79b3c3-7630-44cf-9ec0-62d827eb7709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33f007cbc1814baea8b60d8830cea9c4, user_id=0445562b148a44779be81a2eeece6097, role_id=e60a0103aa774cff9ee9b873bca64fb6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:27.796670 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79b3c3-7630-44cf-9ec0-62d827eb7709 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:27.808036 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1445/2874] 10.222.0.43 () {60 vars in 1482 bytes} [Mon Jul 4 08:13:27 2022] PUT /identity/v3/projects/33f007cbc1814baea8b60d8830cea9c4/users/0445562b148a44779be81a2eeece6097/roles/e60a0103aa774cff9ee9b873bca64fb6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:13:32.747533 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1a877-5d82-4c03-b8c3-52eb14b14168 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:32.748260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1a877-5d82-4c03-b8c3-52eb14b14168 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:32.748399 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1a877-5d82-4c03-b8c3-52eb14b14168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:32.788856 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4ca1a877-5d82-4c03-b8c3-52eb14b14168 None None] MFA Rules not processed for user `0445562b148a44779be81a2eeece6097`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:13:32.790548 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4ca1a877-5d82-4c03-b8c3-52eb14b14168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:32.870864 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1430/2875] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:13:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:13:32.885847 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79b3c3-7630-44cf-9ec0-62d827eb7709 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:32.887915 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8d79b3c3-7630-44cf-9ec0-62d827eb7709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:32.951679 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79b3c3-7630-44cf-9ec0-62d827eb7709 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:32.955441 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0337456-d64a-4252-bb79-3051a305fd62 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a122422a45446c9a63d67b4f931fa47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:32.956110 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0337456-d64a-4252-bb79-3051a305fd62 service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:32.956396 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0337456-d64a-4252-bb79-3051a305fd62 service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:32.956672 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0337456-d64a-4252-bb79-3051a305fd62 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:32.986099 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:13:33.030428 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0337456-d64a-4252-bb79-3051a305fd62 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:33.032062 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0337456-d64a-4252-bb79-3051a305fd62 service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:33.045720 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1446/2876] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:13:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:13:33.185610 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62e07f-1bae-4bd5-a5ac-41df7a1d0504 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:33.185990 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62e07f-1bae-4bd5-a5ac-41df7a1d0504 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:33.186498 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62e07f-1bae-4bd5-a5ac-41df7a1d0504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:33.209438 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4d62e07f-1bae-4bd5-a5ac-41df7a1d0504 None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:13:33.211025 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4d62e07f-1bae-4bd5-a5ac-41df7a1d0504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:33.220379 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:13:33.294180 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1431/2877] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:13:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:13:33.325491 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0337456-d64a-4252-bb79-3051a305fd62 service glance] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:33.327284 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b0337456-d64a-4252-bb79-3051a305fd62 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:33.412788 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0337456-d64a-4252-bb79-3051a305fd62 service glance] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:33.420065 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a6c00-a595-4075-84a3-34b786186dec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:33.421087 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a6c00-a595-4075-84a3-34b786186dec service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:33.421534 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a6c00-a595-4075-84a3-34b786186dec service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:33.422010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a6c00-a595-4075-84a3-34b786186dec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:33.433177 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a6c00-a595-4075-84a3-34b786186dec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:33.436053 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a6c00-a595-4075-84a3-34b786186dec service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:33.448309 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1447/2878] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:13:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 08:13:43.285907 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=20 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:13:43.442295 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=48 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:13:43.445708 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3faa21-4fef-467c-ae39-cf1b468d631f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:43.446045 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3faa21-4fef-467c-ae39-cf1b468d631f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:43.446326 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3faa21-4fef-467c-ae39-cf1b468d631f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:43.465049 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6c3faa21-4fef-467c-ae39-cf1b468d631f None None] MFA Rules not processed for user `c11b4c72af804f68ad2bc19e91f11129`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:13:43.466327 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6c3faa21-4fef-467c-ae39-cf1b468d631f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:43.473980 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1432/2879] 10.222.0.43 () {58 vars in 957 bytes} [Mon Jul 4 08:13:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 08:13:43.517657 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a6c00-a595-4075-84a3-34b786186dec service swift] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:43.529291 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a6c00-a595-4075-84a3-34b786186dec service swift] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:43.533892 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17115bc8-daef-4253-8494-d235974a798c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a122422a45446c9a63d67b4f931fa47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:43.534566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17115bc8-daef-4253-8494-d235974a798c service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:43.534853 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17115bc8-daef-4253-8494-d235974a798c service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:43.535115 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17115bc8-daef-4253-8494-d235974a798c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:43.537067 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-17115bc8-daef-4253-8494-d235974a798c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:43.613698 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17115bc8-daef-4253-8494-d235974a798c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:43.615224 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17115bc8-daef-4253-8494-d235974a798c service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:43.627618 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1448/2880] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:13:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:13:44.228839 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3faa21-4fef-467c-ae39-cf1b468d631f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:44.231503 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6c3faa21-4fef-467c-ae39-cf1b468d631f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:44.319434 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3faa21-4fef-467c-ae39-cf1b468d631f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:44.326424 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a85cf6-67fb-47db-99cb-2c5959346a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:44.327769 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a85cf6-67fb-47db-99cb-2c5959346a22 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:44.328154 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a85cf6-67fb-47db-99cb-2c5959346a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:44.328738 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a85cf6-67fb-47db-99cb-2c5959346a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:44.331732 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-66a85cf6-67fb-47db-99cb-2c5959346a22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:44.351446 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a85cf6-67fb-47db-99cb-2c5959346a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:44.353184 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a85cf6-67fb-47db-99cb-2c5959346a22 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:44.366328 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1433/2881] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:13:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:13:45.738893 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff54d1f-bdc6-43bd-9f26-683b58b4c990 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:45.739515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff54d1f-bdc6-43bd-9f26-683b58b4c990 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:45.739515 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff54d1f-bdc6-43bd-9f26-683b58b4c990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:45.761288 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cff54d1f-bdc6-43bd-9f26-683b58b4c990 None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:13:45.762601 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cff54d1f-bdc6-43bd-9f26-683b58b4c990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:13:45.771427 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1449/2882] 10.222.0.43 () {58 vars in 1009 bytes} [Mon Jul 4 08:13:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:13:45.784260 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a85cf6-67fb-47db-99cb-2c5959346a22 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:13:45.798167 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a85cf6-67fb-47db-99cb-2c5959346a22 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:13:45.804428 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395b1c0-e376-46ed-ac04-398df47d0288 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:13:45.805370 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395b1c0-e376-46ed-ac04-398df47d0288 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:13:45.805669 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395b1c0-e376-46ed-ac04-398df47d0288 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:13:45.805961 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395b1c0-e376-46ed-ac04-398df47d0288 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:13:45.816076 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395b1c0-e376-46ed-ac04-398df47d0288 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:13:45.817801 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395b1c0-e376-46ed-ac04-398df47d0288 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:13:45.827289 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1434/2883] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:13:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 08:13:55.755183 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=37 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:13:55.823461 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=35 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:14:13.073792 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff54d1f-bdc6-43bd-9f26-683b58b4c990 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:14:13.091037 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff54d1f-bdc6-43bd-9f26-683b58b4c990 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:14:13.097377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efaacf-b9f9-44dc-875f-accd23c805e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:14:13.098590 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efaacf-b9f9-44dc-875f-accd23c805e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:14:13.099120 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efaacf-b9f9-44dc-875f-accd23c805e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:14:13.099643 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efaacf-b9f9-44dc-875f-accd23c805e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:14:13.115247 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efaacf-b9f9-44dc-875f-accd23c805e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:14:13.116790 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efaacf-b9f9-44dc-875f-accd23c805e3 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:14:13.130339 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1450/2884] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:14:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 08:14:15.003500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395b1c0-e376-46ed-ac04-398df47d0288 service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:14:15.015897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395b1c0-e376-46ed-ac04-398df47d0288 service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:14:15.021286 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f54c7d-4f7d-4fb3-a163-f04082899e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:14:15.022032 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f54c7d-4f7d-4fb3-a163-f04082899e38 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:14:15.022317 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f54c7d-4f7d-4fb3-a163-f04082899e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:14:15.022623 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f54c7d-4f7d-4fb3-a163-f04082899e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:14:15.024599 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-81f54c7d-4f7d-4fb3-a163-f04082899e38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:14:15.038018 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f54c7d-4f7d-4fb3-a163-f04082899e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:14:15.040873 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f54c7d-4f7d-4fb3-a163-f04082899e38 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:14:15.055185 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1435/2885] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:14:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:14:23.123378 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:14:25.051103 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:14:51.274473 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efaacf-b9f9-44dc-875f-accd23c805e3 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:14:51.276126 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-81efaacf-b9f9-44dc-875f-accd23c805e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:14:51.293642 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efaacf-b9f9-44dc-875f-accd23c805e3 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:14:51.300741 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077151b-62ee-47e5-b744-2fe469a75a28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:14:51.301780 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077151b-62ee-47e5-b744-2fe469a75a28 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:14:51.302124 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077151b-62ee-47e5-b744-2fe469a75a28 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:14:51.302487 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077151b-62ee-47e5-b744-2fe469a75a28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:14:51.304988 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3077151b-62ee-47e5-b744-2fe469a75a28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:14:51.321056 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077151b-62ee-47e5-b744-2fe469a75a28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:14:51.323047 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077151b-62ee-47e5-b744-2fe469a75a28 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:14:51.340255 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1451/2886] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:14:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:14:54.870287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f54c7d-4f7d-4fb3-a163-f04082899e38 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:14:54.872429 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-81f54c7d-4f7d-4fb3-a163-f04082899e38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:14:54.892638 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f54c7d-4f7d-4fb3-a163-f04082899e38 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:14:54.899240 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcb48d-7112-4ff2-ac54-68fa3da97a5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15211e42ec9d4c1699d877f0b55ce918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:14:54.900421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcb48d-7112-4ff2-ac54-68fa3da97a5f service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:14:54.901452 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcb48d-7112-4ff2-ac54-68fa3da97a5f service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:14:54.901910 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcb48d-7112-4ff2-ac54-68fa3da97a5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:14:54.905308 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-09fcb48d-7112-4ff2-ac54-68fa3da97a5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:14:54.924664 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcb48d-7112-4ff2-ac54-68fa3da97a5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:14:54.926897 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcb48d-7112-4ff2-ac54-68fa3da97a5f service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:14:54.942175 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1436/2887] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:14:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:15:01.332165 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:15:04.936490 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:15:06.442887 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077151b-62ee-47e5-b744-2fe469a75a28 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:15:06.454454 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077151b-62ee-47e5-b744-2fe469a75a28 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:15:06.459566 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeb61f-92c6-4d60-af14-451c6d0d34e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '429e173890b84ee289abc9793794ddb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:15:06.460271 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeb61f-92c6-4d60-af14-451c6d0d34e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:15:06.460549 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeb61f-92c6-4d60-af14-451c6d0d34e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:15:06.460821 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeb61f-92c6-4d60-af14-451c6d0d34e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:15:06.463163 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ffdeb61f-92c6-4d60-af14-451c6d0d34e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:15:06.542377 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdeb61f-92c6-4d60-af14-451c6d0d34e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:15:06.543998 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdeb61f-92c6-4d60-af14-451c6d0d34e8 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:15:06.557846 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1452/2888] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:15:13.320522 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcb48d-7112-4ff2-ac54-68fa3da97a5f service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:15:13.322126 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-09fcb48d-7112-4ff2-ac54-68fa3da97a5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:15:13.335998 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcb48d-7112-4ff2-ac54-68fa3da97a5f service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:15:13.341946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:15:13.341946 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:15:13.342223 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:15:13.342492 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] PATH_INFO: `/v3/users/fa28f0e0b3b0488faf7db7bf7dc3854b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:15:13.343695 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa28f0e0b3b0488faf7db7bf7dc3854b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:15:13.346014 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:15:13.407053 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa28f0e0b3b0488faf7db7bf7dc3854b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.407752 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa28f0e0b3b0488faf7db7bf7dc3854b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.408437 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa28f0e0b3b0488faf7db7bf7dc3854b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.418385 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa28f0e0b3b0488faf7db7bf7dc3854b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.432034 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa28f0e0b3b0488faf7db7bf7dc3854b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.447840 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1437/2889] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:15:13 2022] DELETE /identity/v3/users/fa28f0e0b3b0488faf7db7bf7dc3854b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:15:13.668954 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeb61f-92c6-4d60-af14-451c6d0d34e8 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:15:13.680470 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeb61f-92c6-4d60-af14-451c6d0d34e8 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:15:13.760784 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:15:13.761721 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:15:13.761980 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:15:13.762257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] PATH_INFO: `/v3/projects/d1b0f9c7bc044f6c802c8cd34da4eb0e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:15:13.763465 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1b0f9c7bc044f6c802c8cd34da4eb0e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:15:13.765394 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:15:13.820022 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1b0f9c7bc044f6c802c8cd34da4eb0e', 'request_id': 'req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.825595 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1b0f9c7bc044f6c802c8cd34da4eb0e', 'request_id': 'req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.834718 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1b0f9c7bc044f6c802c8cd34da4eb0e', 'request_id': 'req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.847450 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] The token cache is being invalidate because project d1b0f9c7bc044f6c802c8cd34da4eb0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:15:13.847738 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.848440 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.850040 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1453/2890] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:15:13 2022] DELETE /identity/v3/projects/d1b0f9c7bc044f6c802c8cd34da4eb0e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 08:15:13.854169 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:15:13.855608 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:15:13.869421 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279d1e3-b9ac-4c0c-88a3-6c906dd1bad1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:15:13.930019 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed8d26-2229-49c9-a95a-8996d71bd147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:15:13.930918 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed8d26-2229-49c9-a95a-8996d71bd147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:15:13.931202 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed8d26-2229-49c9-a95a-8996d71bd147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:15:13.931478 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed8d26-2229-49c9-a95a-8996d71bd147 admin admin] PATH_INFO: `/v3/users/8572f7d0b16f4c20abb90f5af9814609` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:15:13.932806 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed8d26-2229-49c9-a95a-8996d71bd147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8572f7d0b16f4c20abb90f5af9814609)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:15:13.934854 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed8d26-2229-49c9-a95a-8996d71bd147 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:15:13.989257 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c3ed8d26-2229-49c9-a95a-8996d71bd147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8572f7d0b16f4c20abb90f5af9814609'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.989983 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c3ed8d26-2229-49c9-a95a-8996d71bd147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8572f7d0b16f4c20abb90f5af9814609'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.990601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c3ed8d26-2229-49c9-a95a-8996d71bd147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8572f7d0b16f4c20abb90f5af9814609'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:13.997749 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c3ed8d26-2229-49c9-a95a-8996d71bd147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8572f7d0b16f4c20abb90f5af9814609'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:14.010391 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c3ed8d26-2229-49c9-a95a-8996d71bd147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8572f7d0b16f4c20abb90f5af9814609'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:14.012114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:15:14.031114 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1438/2891] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:15:13 2022] DELETE /identity/v3/users/8572f7d0b16f4c20abb90f5af9814609 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:15:14.233120 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:15:14.234647 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:15:14.249402 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4c64c-f441-4b5c-a955-10db2f2ecacd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:15:14.273743 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:15:14.316288 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:15:14.317153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:15:14.317430 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:15:14.317732 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] PATH_INFO: `/v3/projects/1f91023a8d9e47c0b8eb1e8288abd7a3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:15:14.318986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f91023a8d9e47c0b8eb1e8288abd7a3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:15:14.320934 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:15:14.375459 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f91023a8d9e47c0b8eb1e8288abd7a3', 'request_id': 'req-063a615e-60fe-404a-acde-224bbbbb0526'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:14.382422 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f91023a8d9e47c0b8eb1e8288abd7a3', 'request_id': 'req-063a615e-60fe-404a-acde-224bbbbb0526'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:14.391795 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f91023a8d9e47c0b8eb1e8288abd7a3', 'request_id': 'req-063a615e-60fe-404a-acde-224bbbbb0526'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:14.403810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] The token cache is being invalidate because project 1f91023a8d9e47c0b8eb1e8288abd7a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:15:14.404103 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:14.404786 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:14.407093 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1454/2892] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:15:14 2022] DELETE /identity/v3/projects/1f91023a8d9e47c0b8eb1e8288abd7a3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:15:24.030060 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:15:24.400519 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:15:34.350268 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ca6a0-64aa-47ca-b09b-f1e8af1aaba7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:15:34.350782 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ca6a0-64aa-47ca-b09b-f1e8af1aaba7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:15:34.350898 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ca6a0-64aa-47ca-b09b-f1e8af1aaba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:15:34.370972 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b94ca6a0-64aa-47ca-b09b-f1e8af1aaba7 None None] MFA Rules not processed for user `80a45a063c0e4f1e9b2156fb2f4cd8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 08:15:34.372287 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b94ca6a0-64aa-47ca-b09b-f1e8af1aaba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:15:34.437435 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1439/2893] 10.222.0.43 () {58 vars in 967 bytes} [Mon Jul 4 08:15:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 08:15:34.479149 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:15:34.480678 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:15:34.556249 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a615e-60fe-404a-acde-224bbbbb0526 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:15:34.562319 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e333aaa-9d8b-4c31-a9f5-e68a6bebd609 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6763884b51204c488ca8c7e452e3a29e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:15:34.562981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e333aaa-9d8b-4c31-a9f5-e68a6bebd609 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:15:34.563254 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e333aaa-9d8b-4c31-a9f5-e68a6bebd609 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:15:34.563525 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e333aaa-9d8b-4c31-a9f5-e68a6bebd609 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:15:34.573085 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e333aaa-9d8b-4c31-a9f5-e68a6bebd609 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:15:34.575538 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e333aaa-9d8b-4c31-a9f5-e68a6bebd609 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:15:34.588849 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1455/2894] 10.222.0.43 () {60 vars in 1384 bytes} [Mon Jul 4 08:15:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 08:15:38.363285 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ca6a0-64aa-47ca-b09b-f1e8af1aaba7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:15:38.365275 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b94ca6a0-64aa-47ca-b09b-f1e8af1aaba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:15:38.487562 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ca6a0-64aa-47ca-b09b-f1e8af1aaba7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:15:38.493963 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33389dc4-dc82-4954-89bb-aa22a93cde7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57dccfe9f5f4bc397f4c470fbf047dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:15:38.495039 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33389dc4-dc82-4954-89bb-aa22a93cde7a service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:15:38.495500 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33389dc4-dc82-4954-89bb-aa22a93cde7a service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:15:38.495964 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33389dc4-dc82-4954-89bb-aa22a93cde7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:15:38.498722 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-33389dc4-dc82-4954-89bb-aa22a93cde7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:15:38.513871 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33389dc4-dc82-4954-89bb-aa22a93cde7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:15:38.515453 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33389dc4-dc82-4954-89bb-aa22a93cde7a service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:15:38.528714 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1440/2895] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:15:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:15:41.555854 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e333aaa-9d8b-4c31-a9f5-e68a6bebd609 service swift] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:15:41.558010 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0e333aaa-9d8b-4c31-a9f5-e68a6bebd609 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:15:41.641709 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e333aaa-9d8b-4c31-a9f5-e68a6bebd609 service swift] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:15:41.649018 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:15:41.650072 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:15:41.650359 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:15:41.650631 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] PATH_INFO: `/v3/users/c11b4c72af804f68ad2bc19e91f11129` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:15:41.651979 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c11b4c72af804f68ad2bc19e91f11129)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:15:41.653983 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:15:41.712793 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c11b4c72af804f68ad2bc19e91f11129'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:41.721913 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c11b4c72af804f68ad2bc19e91f11129'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:41.729096 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c11b4c72af804f68ad2bc19e91f11129'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:41.729777 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c11b4c72af804f68ad2bc19e91f11129'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:41.742810 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c11b4c72af804f68ad2bc19e91f11129'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:41.750770 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1456/2896] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:15:41 2022] DELETE /identity/v3/users/c11b4c72af804f68ad2bc19e91f11129 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:15:41.945267 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33389dc4-dc82-4954-89bb-aa22a93cde7a service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:15:41.956375 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33389dc4-dc82-4954-89bb-aa22a93cde7a service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:15:42.017226 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:15:42.018131 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:15:42.018405 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:15:42.018675 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] PATH_INFO: `/v3/projects/fea21868e80c43aa81d1a9d277da4e9d` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:15:42.019913 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea21868e80c43aa81d1a9d277da4e9d)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:15:42.022084 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:15:42.061601 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:15:42.071423 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea21868e80c43aa81d1a9d277da4e9d', 'request_id': 'req-01318b34-af99-445c-ae82-411b68942b99'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.076804 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea21868e80c43aa81d1a9d277da4e9d', 'request_id': 'req-01318b34-af99-445c-ae82-411b68942b99'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.083278 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea21868e80c43aa81d1a9d277da4e9d', 'request_id': 'req-01318b34-af99-445c-ae82-411b68942b99'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.097523 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] The token cache is being invalidate because project fea21868e80c43aa81d1a9d277da4e9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:15:42.097847 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.098634 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.100344 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1441/2897] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:15:41 2022] DELETE /identity/v3/projects/fea21868e80c43aa81d1a9d277da4e9d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:15:42.105286 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:15:42.106986 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:15:42.122153 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88d438-aee4-4b9e-9628-3e893f7aa9b9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:15:42.128052 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:15:42.190367 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:15:42.191257 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:15:42.191535 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:15:42.191811 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] PATH_INFO: `/v3/users/0445562b148a44779be81a2eeece6097` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:15:42.193326 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=0445562b148a44779be81a2eeece6097)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:15:42.195332 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:15:42.250981 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0445562b148a44779be81a2eeece6097'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.263485 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0445562b148a44779be81a2eeece6097'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.274038 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0445562b148a44779be81a2eeece6097'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.275129 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0445562b148a44779be81a2eeece6097'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.292649 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0445562b148a44779be81a2eeece6097'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.299937 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1457/2898] 10.222.0.43 () {58 vars in 1223 bytes} [Mon Jul 4 08:15:42 2022] DELETE /identity/v3/users/0445562b148a44779be81a2eeece6097 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:15:42.480881 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:15:42.482929 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:15:42.496532 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01318b34-af99-445c-ae82-411b68942b99 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:15:42.565007 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91712d-f44e-4b62-910f-9447101db8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec7b27e85c74490a82ad099b2e0743e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3a0790796c49f294035007b8a8c25a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:15:42.565934 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91712d-f44e-4b62-910f-9447101db8aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:15:42.566200 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91712d-f44e-4b62-910f-9447101db8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:15:42.566475 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91712d-f44e-4b62-910f-9447101db8aa admin admin] PATH_INFO: `/v3/projects/33f007cbc1814baea8b60d8830cea9c4` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:15:42.567891 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91712d-f44e-4b62-910f-9447101db8aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=33f007cbc1814baea8b60d8830cea9c4)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:15:42.569899 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91712d-f44e-4b62-910f-9447101db8aa admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:15:42.620501 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-da91712d-f44e-4b62-910f-9447101db8aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33f007cbc1814baea8b60d8830cea9c4', 'request_id': 'req-da91712d-f44e-4b62-910f-9447101db8aa'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.625969 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-da91712d-f44e-4b62-910f-9447101db8aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33f007cbc1814baea8b60d8830cea9c4', 'request_id': 'req-da91712d-f44e-4b62-910f-9447101db8aa'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.632583 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-da91712d-f44e-4b62-910f-9447101db8aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33f007cbc1814baea8b60d8830cea9c4', 'request_id': 'req-da91712d-f44e-4b62-910f-9447101db8aa'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.648805 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-da91712d-f44e-4b62-910f-9447101db8aa admin admin] The token cache is being invalidate because project 33f007cbc1814baea8b60d8830cea9c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 08:15:42.649147 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-da91712d-f44e-4b62-910f-9447101db8aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.650145 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-da91712d-f44e-4b62-910f-9447101db8aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 08:15:42.652488 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1442/2899] 10.222.0.43 () {58 vars in 1232 bytes} [Mon Jul 4 08:15:42 2022] DELETE /identity/v3/projects/33f007cbc1814baea8b60d8830cea9c4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 08:15:52.295565 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:15:52.643822 dv-cinder-797608-42-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 08:17:33.150242 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 08:17:33.152941 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:17:33.319438 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960333c2-bc7d-468d-86da-fc2b611cd7ef admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 08:17:33.397746 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbffe6-3765-470c-9ab5-25c8d3477054 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57dccfe9f5f4bc397f4c470fbf047dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f2d85c9e3d14bdabfb63fe505d83714', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 08:17:33.398321 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbffe6-3765-470c-9ab5-25c8d3477054 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 08:17:33.398572 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbffe6-3765-470c-9ab5-25c8d3477054 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 08:17:33.398880 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbffe6-3765-470c-9ab5-25c8d3477054 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 08:17:33.400889 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-64bbffe6-3765-470c-9ab5-25c8d3477054 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 08:17:33.412840 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbffe6-3765-470c-9ab5-25c8d3477054 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 08:17:33.414603 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbffe6-3765-470c-9ab5-25c8d3477054 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 08:17:33.425406 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1458/2900] 10.222.0.43 () {60 vars in 1365 bytes} [Mon Jul 4 08:17:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 08:17:43.420627 dv-cinder-797608-42-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}